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FOREWORD 


ePIRLS: An International Assessment 
of Reading for New Times 


Donald J. Leu, Ph.D. 
Director, New Literacies Research Lab and Professor of Education 
University of Connecticut 


We change the world when a child learns to read. Learning to read opens windows to the world and 
new opportunities for everyone. Despite our best efforts, far too many lack access to high quality 
reading instruction and many students continue to struggle, increasing the possibility that they will 
drop out of school. It is essential for all students to become fully prepared in reading so that they are 
able to succeed in school, fulfill individual goals, and make our world a better place through their 
accomplishments. Thus, reading has long been universally recognized as a core area of educational 
policy and instruction for every nation. Today, the Internet has changed what it means to become a 
reader. New educational policies and practices, based on data, are required to adapt to these changes. 


The Changing Nature of Reading 


The Internet is an inherently disruptive information and communication technology that has 
changed lives in profound ways in every nation. These changes affect both the nature of reading 
and the sources of information that we use for learning (Le Bigot & Rouet, 2007). In addition to 
being able to read traditional texts, we now require the skills, strategies, and practices that enable us 
to read and learn online. In response to this changing dynamic, IEA’s TIMSS & PIRLS International 
Study Center has developed a new assessment of online reading, ePIRLS. 

ePIRLS is a performance-based assessment of students’ ability to read and learn online with 
school-based assignments in science and social studies. The ePIRLS assessment tasks are delivered 
on a computer and include lessons with webpages, information graphics, animations, multiple tabs, 
pop-up windows, and an avatar that guides students through the research tasks. The results from this 
new assessment of online reading and learning provide educators with needed information to guide 
policy in a rapidly changing area. Fourteen countries and two benchmarking participants joined 
ePIRLS 2016. As a result, they now have important data on how well their fourth grade students are 
prepared for the demands of online informational reading and learning. 
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The focus on students in their fourth year of schooling is a wise one, especially for the evaluation 
of online reading. A reasonably accurate aphorism that applies to reading is: “We learn to read in the 
first three to four years of school and we read to learn in subsequent years.” By focusing on fourth 
grade students, ePIRLS is the only international assessment to provide us with data situated at this 
crucial, developmental nexus for reading. The grade cohort is perfectly positioned to inform us 
about both aspects of reading development, especially preparation for the important task of reading 
to learn from online sources. 

In addition to a new assessment of online reading and learning, ePIRLS also includes surveys 
of teachers and school leaders in participating countries. This triad of data provides policy makers 
with information essential to developing appropriate public policies in this new area of reading, 
thereby improving the well-being of their citizens in a digital age of online information. 

Why should anyone invest a tremendous amount of effort, time, and resources to develop and 
use an assessment of online reading? There are many reasons. 


1. First, between 40 and 50 percent of the world’s population currently has access to the 
Internet (UNESCO, 2014). At the current rate of adoption, all, or nearly all, of the 
world’s population will have access in just eight more years. This means that children who 
are in fourth grade today will graduate from secondary school and enter a universal world 
of online information. To adequately prepare young students for this future requires that we 
gather information about the nature of their progress with reading in online contexts. This 
enables us to prepare them for the world of online information they will inherit. 


2. Second, students have increasing access to online information at home and on mobile 
devices and they use these often. Data show that students in some countries spend more 
time on computing devices than in books. Outside of school, students in the United States 
from ages 8 to 18 spend three times more time reading on a computing device than they 
spend reading traditionally printed pages offline (Rideout, Foehr, & Roberts, 2010). This 
and other developing trends in online reading (see, for example, Braten, McCrudden, 
Lund, Brante, & Stomsg, 2017; Leu, et al., 2016) present challenges for educators as we look 
to support learners in a digital age (cf. Goodman, Sands, & Coley, 2015; Kirsch, Braun, 
Yamamoto, & Sum, 2007; Larson & Dwyer, 2015). Students are telling us what their reading 
lives are like so nations and classrooms can either adapt to this new reality or else become 
less relevant to the lives of this generation. 


3. Third, students are unskilled with reading information online to learn. With the 
development of ePIRLS, we recognize the need to plan for high levels of critical thinking 
in the reading of online texts, beginning with younger readers. Although today’s students 
grow up in an online world and are developing skills in gaming, social networking, media 
creation, and texting, research is showing how limited students’ skills are with online 
reading. They are not skilled at locating information online (Bilal, 2000; Guinee, Eagleton, 
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& Hall, 2003; Kuiper & Volman, 2008) or critically evaluating it (Walraven, Brand-Gruwel, 
& Boshuizen, 2008). Many students find it difficult to judge the accuracy, reliability, and 
bias of information that they encounter during online research (Bennett, Maton, & Kervin, 
2008; Graham & Metaxas, 2003). In fact, adolescents overgeneralize their ability to read 
and evaluate online information effectively, a perception informed by their ability to engage 
successfully with online social networking, texting, and video games (Kuiper & Volman, 
2008). 


4. Fourth, research indicates that online reading comprehension is not isomorphic with 
offline reading comprehension (Afflerbach & Cho, 2010; Coiro & Dobler, 2007). Thus, 
the argument is no longer sustainable that we do not require additional reading assessments 
because online and offline reading are identical. Recent studies have shown important 
differences between online and offline reading. Afflerbach and Cho reviewed 46 studies 
that focused on reading strategy use during Internet and hypertext reading. Their analysis 
showed evidence of strategies that “appear to have no counterpart in traditional reading” 

(p. 217). Many strategies centered around a reader's ability to apply methods to reduce 

their levels of uncertainty while navigating appropriate reading paths in a shifting problem 
space. Examples include the use of keywords and search engine results during reading and 
problem solving with online information. It also includes critically evaluating the reliability 
of online information using links and strategies not found with traditional text. By gathering 
information about the online reading ability of fourth grade students, we generate greater 
awareness and understanding of these differences and that allows us to introduce classroom 
experiences to develop proficiency in the additional areas required for online reading. 


5. Finally, issues of equity have become increasingly important and a separate online 
reading achievement gap is appearing. ‘This is essential to consider at a time when just 
62 individuals own the same wealth as the lower half of the world’s population (Oxfam, 
2016). If we are to address issues of income inequality, appropriate performance data must 
be collected and equal opportunity for all students to learn to read online must be available 
in schools, even when access is not available at home. Data from ePIRLS will permit these 
important goals to be realized. 


ePIRLS Results: 2016 


The results from the first iteration of ePIRLS are already providing important direction for our work 
ahead in reading. They have provided us with an important new model for the assessment of online 
reading comprehension, one that according to students provides an engaging online simulation 
of school assignments in science and social studies. The use of a performance-based simulation 
is especially innovative and important. According to de Klerk, Veldkamp, and Eggen (2015), 
simulations have at least three advantages over other forms of assessment: engagement increases, 
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thereby increasing flow (Csikszentmihalyi, 1991) and avoiding anxiety; knowledge application, 
rather than simple knowledge replication, is emphasized; and richer data can be captured, including 
process data. 

A simulation of online reading and learning has another important function. ePIRLS permits 
educators to see what online reading looks like in action and shows them the additional new skills 
that may be needed by students for successful learning. This will play an essential role in helping 
teachers to understand the nature of online reading and assist them in expanding the nature of 
classroom reading instruction to include the development of these skills. 

Early analysis indicates a number of ePIRLS results are important to consider. These include: 


e Students liked taking the assessments. The percentage of students who liked each 
assessment activity ranged from 83 to 93 percent. This may provide additional support 
for the use of simulations during assessment since we are able to assess students’ optimal 
performance only when they are engaged. 


e Students had little difficulty in managing the assessment. Students were able to navigate 
to a high percentage of the items, with almost all completing the assessment in the time 
allotted. Students reported a high degree of self-efficacy in computer use. 


e Countries that reported higher mean ePIRLS scores compared to PIRLS scores were 
countries that traditionally do better with assessments of offline reading. This suggests 
that the benefits of good instructional contexts are likely to generalize to both types of 
reading. 


e Girls appear to do better than boys in ePIRLS at both levels of comprehension. This 
suggests that technology advantages, often enjoyed by boys, may not apply to reading and 
learning with online information. There was no country where boys performed higher than 
girls in ePIRLS. 


ePIRLS: New Opportunities and New Challenges 


With the development of ePIRLS, combined with PIRLS, we now have a comprehensive set of tools 
to measure reading on an international scale and at an age point especially important to reading 
development, the fourth grade of schooling. The importance of this accomplishment needs to be 
highlighted. Being able to provide comprehensive information about reading performance in both 
online and offline contexts, at an important developmental point, is a major achievement and will 
continue to benefit policy decisions for nations. Thus, those countries participating in ePIRLS may 
be doubly advantaged in reading today as students must navigate in both online and offline reading 
contexts. 

ePIRLS has also pointed us to new opportunities as nations consider ways in which to 
encourage greater uptake in technology and the sciences by girls and women. We have known that 
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girls generally perform at high levels and are highly motivated by reading. Finding that this pattern 
also appears in the reading of online information, especially in science, may provide us with a new 
direction for thinking about the delivery of science instruction in ways that support high levels of 
performance and engagement by girls. This would enable many nations to enhance their scientific 
workforce in important ways. 

ePIRLS also points us to new opportunities to understand optimal patterns of classroom 
instruction and home use of the Internet at a crucial period in the development of children. As we 
begin to understand these relationships better, we will be able to craft even more insightful means to 
measure instruction in online reading and home use of the Internet, providing valuable information 
about how best to support development. 

An especially important opportunity has been made possible with the development of 
a performance-based assessment in this area, which has provided us with a new model for 
the assessment of reading. This will likely lead to new approaches to the assessment of student 
learning. Especially important in this regard is the observation that students, both boys and girls, 
liked working on the ePIRLS tasks. By sustaining high levels of enjoyment during assessment we 
are able to evaluate optimal levels of performance. 

There are several challenges that also come with the changing nature of reading and reading 
assessment. Recent events in the world remind us of the special importance of being able to read 
critically and evaluate online information carefully. The Internet includes a diverse set of voices 
and lacks traditional gatekeepers for information. As a result, it challenges readers by requiring an 
especially high level of critical evaluation. 

Our inability to evaluate online information has already been demonstrated to have serious 
political, social, and economic consequences. Thus, it is important that we continue to consider 
tracking the important area of students’ ability to critically evaluate the reliability of online 
information. 

In addition, we will be challenged by the changes to reading that will continue to take 
place. The Internet changes the nature of what it means to read and learn online and it does so 
continuously. We have already seen that to be literate yesterday, in a world defined primarily by static 
book technologies, does not ensure that one is fully literate today. To be literate tomorrow will be 
defined by even newer technologies that have yet to appear and the new skills, strategies, and social 
practices these will require. ePIRLS has stepped boldly into this new, continually changing context, 
providing us with essential information about our future, the children who are in school today. It is 
a remarkable achievement and will be remembered as an important historical milestone. 
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About ePIRLS 2016 


Overview 


The Internet has become the primary source for obtaining information at work, at home, and for 
school. Because Internet reading increasingly is becoming one of the central ways students are 
acquiring information, in 2016, PIRLS was extended to include ePIRLS—an innovative assessment 
of online reading. ePIRLS is a computer-based assessment that uses an engaging, simulated Internet 
environment to present fourth grade students with authentic school-like assignments involving 
science and social studies topics. For examples, see Take the ePIRLS Assessment. An Internet 
browser window provides students with a website containing information about their assignments, 
and students navigate through pages with a variety of features, such as graphics, multiple tabs, links, 
pop-up windows, and animation. In an assessment window, a teacher avatar guides students through 
the ePIRLS assignments, prompting the students with questions about the online information. 

Participating in PIRLS 2016 was a prerequisite for participating in ePIRLS, so that the countries 
and students participating in ePIRLS are subsets of those that participated in PIRLS 2016 (see 
About PIRLS). Like PIRLS 2016, the ePIRLS assessment was developed based on the PIRLS 2016 
Assessment Framework, used the same quality assurance procedures, and was given to the same 
students who participated in the PIRLS assessment typically on the next day. Thus, as an extension of 
PIRLS, ePIRLS results can be considered in the context of the PIRLS results, including comparative 
achievement on PIRLS and in relation to the PIRLS context questionnaire data. 

TIMSS and PIRLS are directed by IEAs TIMSS & PIRLS International Study Center at Boston 
College in close cooperation with IEA Amsterdam, IEA Hamburg, and Statistics Canada. IEA is an 
independent international cooperative of national research institutions and government agencies 
that pioneered international assessments of student achievement in the 1960s to gain a deeper 
understanding of policy effects across countries’ different systems. IEA has been conducting 
international assessments of reading literacy and the factors associated with proficient reading 
comprehension in countries around the world for about 60 years. 


The ePIRLS Tasks 


The ePIRLS assessment consisted of five tasks with each task lasting up to 40 minutes. Each student 
was asked to complete two of the tasks according to a specific rotation plan. The assessments were 
administered via computer (typically PCs) and students entered their answers by clicking on options 
or typing words. 
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Because the idea of ePIRLS is new, two of the tasks are available on the reports website 
website—“Mars” and “Elizabeth Blackwell (the first woman doctor).” The ePIRLS 2016 Example 
Tasks video provides an overview of the two tasks, and Take the ePIRLS Assessment provides the 


two tasks in their entirety, so they can be viewed in the same way they were given to students. You 
can enter your answers and the scoring key is provided. 


ePIRLS 2016 Results 


The international results for ePIRLS are reported on the reports website and the results for PIRLS 
2016 also can be accessed from here. The ePIRLS 2016 International Results in Online Informational 
Reading includes four chapters or sections providing numerous exhibits summarizing student 
achievement on ePIRLS compared to PIRLS overall and at the PIRLS 2016 International Benchmarks. 
Results also are presented in relation to students’ home and school contexts for learning to read 


online. The exhibits can be downloaded and printed from the Download Center. 


ABOUT ePIRLS 2016 Z TIMSS & PIRLS 
—) International Study Center 4 
ePIRLS 2016 INTERNATIONAL RESULTS IN ONLINE INFORMATIONAL READING Zw IE ieynziScheoForducation 


Downloaded from http://pirls2016.org/download-center/ BOSTON COLLEGE 


ePIRLS 


Online 


Reading 


2016 Grade 


Exhibit 1 shows the ePIRLS 2016 countries and benchmarking participants, including 14 countries 


and 2 benchmarking entities. 


ePIRLS 


Online 


Reading 


-xhibit 1: Countries Participating in ePIRLS 2016 


Canada 
Chinese Taipei 
Denmark 
Georgia 
lreland 

Israel 

Italy 


Norway (5) Benchme 
Portugal 

Singapore Abu Dhabi, UAE 
Slovenia Dubai, UAE 
Sweden 


United Arab Emirates 
United States 


Note: Norway chose to assess the fifth grade to obtain better comparisons with Sweden. 
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Exhibit 2: Percentages of Students Who Liked Working on the ePIRLS Tasks 
Exhibit 2 shows the percentages of students who liked working on the ePIRLS tasks. In general, the 
students participating in ePIRLS liked working on the tasks (89% on average). The tasks were liked 
equally by girls and boys, with the exception of the Dr. Elizabeth Blackwell task (88% for girls and 
78% for boys). 


Exhibit 2: Percentages of Students Who Liked Working on the ePIRLS Tasks 


Students’ Reports 
Note: Results based on students who participated in both PIRLS and ePIRLS. 


Percent of Students Who Liked the Task A Lot or A Little 


ePIRLS Task 


Boys 


Mars 88 (0.3) 87 (0.4) 89 (0.3) 
Rainforests 93 (0.2) 94 (0.3) 92 (0.3) 
Dr. Elizabeth Blackwell 83 (0.3) 88 (0.4) 78 (0.5) 
Zebra and Wildebeest Migration 92 (0.2) 93 (0.3) 92 (0.3) 
The Legend of Troy 89 (0.3) 89 (0.3) 90 (0.3) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Example: 


How much did you like working on the project about Mars? 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


| liked it a lot 


| liked it a little 
| didn’t like it very much 


| didn’t like it at all 


SAVE 
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ePIRLS Average Achievement 


Singapore iy) 

Norway Gy Ireland &y 
Sweden@) Denmark United States 
Chinese Taipei Canada ZB) 
Israel) Italy EP 
Slovenia) Portugal >) 


Georgia United Arab Emirates (3) 
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Relative Performance on ePIRLS and PIRLS 
Difference in Achievement Score Points 
PIRLS Score is Higher 


ePIRLS Online Informational 
Score is Higher 


United Arab Emirates 
Singapore 


Denmark 


Relatively Higher Norway 
Achievement 5 
on ePIRLS United States 


Israel 


Sweden Similar Achievement 


Ireland =e on ePIRLS and PIRLS 
poneee Relatively Higher 
Achievement on PIRLS 


Chinese Taipei 


Slovenia 


In ePIRLS, Girls Had Higher Achievement Than Boys 


In 13 countries, girls 


11 Countries 3 Countries performed better at 

Girls Higher Achievement Similar retrieving information 

Peon raBoys and straightforward 

chievement Italy, Portugal, Denmark inferencing 

United States, Canada, : 

Chinese Taipei, Ireland, )\¥o Countries They performed 

Israel, Slovenia, Boys Higher better in interpreting, 

Sweden, Georgia, Achievement integrating, and 

Norway, Singapore, evaluating in 

United Arab Emirates 9 countries. 
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CHAPTER 1 


ePIRLS Student Achievement 


Exhibit 1.1: ePIRLS Average Achievement 


Exhibit 1.1 shows average student achievement for the participants in ePIRLS 2016. The first column 
shows average achievement on the ePIRLS assessment, the second column shows the ePIRLS 
students’ average achievement on PIRLS, and the third column shows the difference between the 
two. The ePIRLS achievement scale summarizes fourth grade students’ reading achievement in a 
simulated online environment where each task was based on a series of interconnected webpages 
with many different kinds of visual information as well as texts. The webpages provided information 
about science and social studies topics, and students were guided through an online study similar to 
the types of projects or reports they might be asked to complete for school. The ePIRLS assessment 
included 5 informational online reading tasks, with 91 items. 

The ePIRLS results are reported on the PIRLS reading achievement scale to facilitate 
comparisons of relative performance between ePIRLS and PIRLS. The resulting ePIRLS scores 
are directly comparable to PIRLS scores, so that students with higher scores on ePIRLS can be 
considered to have performed relatively better than on PIRLS, and students with lower ePIRLS 
scores to have performed relatively less well than on PIRLS. 


The definitive PIRLS 2016 achievement results for PIRLS are shown in Exhibit 1.1 of PIRLS 
2016 International Results in Reading. The PIRLS 2016 results shown in ePIRLS Exhibit 
1.1 are only provided for the purpose of comparing relative performance on PIRLS and 
ePIRLS 2016. 


Although the plan was to have all PIRLS students participate in ePIRLS, this did not occur 
perfectly due to student absences and some issues with the computer equipment. As a result, 
somewhat fewer students participated in ePIRLS than PIRLS. For making comparisons, the PIRLS 
results shown in ePIRLS Exhibit 1.1 are based on only the students that participated in ePIRLS. 

It also should be kept in mind that while ePIRLS assesses how well students can read 
information in an online environment that consists of using content tabs, navigation bars, graphic 
icons, links, and scroll bars, PIRLS is a more general measure of reading comprehension. The PIRLS 
achievement scale summarizes fourth grade students’ performance answering questions designed 
to measure their reading comprehension across two overarching purposes for reading—literary and 
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informational purposes presented on paper in a linear format. PIRLS consisted of 6 literary passages 
with 90 items and 6 informational passages with 85 items. 

Essentially, the students in all the countries participating in ePIRLS were good to excellent 
readers as evidenced by their performance on ePIRLS and PIRLS. Most countries participating 
in ePIRLS had average achievement on ePIRLS and PIRLS that was well above the PIRLS scale 
centerpoint of 500. Still, the ePIRLS achievement results show a range in performance from the 
top-performing to the lower-performing countries. 

Singapore was the top-performing country on ePIRLS followed by Norway and Ireland. Also, 
some countries had relatively higher achievement on ePIRLS than PIRLS, and other countries had 
relatively higher achievement on PIRLS than ePIRLS. The graph included as part of Exhibit 1.1 
indicates which countries had relatively higher achievement on ePIRLS and which had relatively 
higher achievement on PIRLS. Internationally, there was not a predominant pattern one way or the 
other. Singapore, Norway, Sweden, Denmark, the United States, Israel, and the United Arab Emirates 
had higher achievement on ePIRLS, whereas Chinese Taipei, Italy, Slovenia, Portugal, and Georgia 
had higher achievement on PIRLS. Ireland and Canada performed similarly on ePIRLS and PIRLS. 

According to an informal exchange among the National Research Coordinators in the 
participating ePIRLS countries, a relative advantage in ePIRLS average achievement compared to 
PIRLS average achievement may be related to how familiar students are with using computers 
in school contexts, especially as part of classroom instructional activities or in assessment. The 
questionnaire data provide some support for these hypotheses. For example, as a whole, more 
students in the countries with relatively higher achievement on ePIRLS had high access to digital 
devices in the home (Exhibit 3.2) and attended schools not affected by digital resource shortages 
(Exhibit 3.3). Also in those countries, about one-third of the students—even more in Singapore 
(42%) and Israel (55%)—spent at least 30 minutes per day using computers to prepare reports 
(Exhibit 3.4). The exception was Sweden (21%). In the PIRLS 2016 Encyclopedia, all the countries 
with higher relative achievement on ePIRLS (except Norway) reported a priority on using online 
resources and computers in reading instruction. Singapore incorporates a number of non-print 
resources such as web-based texts into the teaching and learning of language and literacy. In the 
United States many literacy and reading textbooks use the Internet and instructional technologies 
as part of reading instruction. 
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Exhibit 1.1: ePIRLS Average Achievement 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Ke) 
: PIRLS z 
Informational P = 
Country Average Scale Difference ePIRLS Online i 
Average Scale PIRLS = 
ee Score Informational score tigher 2 
Score is Higher 9 
3 Singapore 588 (3.0) 576 (3.1) 08) = £ 
Norway (5) 568 (2.2) 560 (2.3) 8 (1.6) Frossese] 2 
Ireland 567 (2.5) 566 (2.8) 1 (1.2) | & 
Sweden 559 (2.3) 555 (2.4) 4 (1.1) = 3 
= Denmark 558 (2.2) 548 (2.3) 11 (1.7) Ss 2 
t United States 557 (2.6) 550 (2.9) 7 (1.2) race 5 
Chinese Taipei 546 (2.0) 559 (2.0) -13 (1.0) = = 
12 Canada 543 (3.2) 543 (3.3) 0 (1.5) | i 
3 Israel 536 (2.3) 532 (2.5) 5 (1.2) = 3 
Italy 532 (2.1) 548 (2.4) -16 (1.7) oes) » 
Slovenia 525 (1.9) 543 (2.0) -18 (1.0) Sey = 
2 Portugal 522 (2.2) 528 (2.3) -5 (1.2) = S 
1 Georgia 477 (3.3) 489 (3.1) -12 (2.1) [=a 3 
United Arab Emirates 468 (2.2) 451 (2.7) 18 (1.2) a 
Benchmarking Participants 
Dubai, UAE 528 (1.6) 516 (1.9) 12 (1.0) = 
Abu Dhabi, UAE 431 (4.1) 414 (4.8) 17 (2.2) Sees 
T 


@ Difference statistically significant 


™ Difference not statistically significant 


To facilitate comparisons of relative performance on ePIRLS and PIRLS, the ePIRLS data are reported on the PIRLS achievement scale. The resulting ePIRLS scores are directly 
comparable to PIRLS scores, so that students with higher scores on ePIRLS can be considered to have performed relatively better than on PIRLS, and students with lower ePIRLS 
scores to have performed relatively less well than on PIRLS. 

See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 

() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 1.2: Multiple Comparisons of Average Achievement 
Because there often were relatively small differences in achievement between countries, Exhibit 
1.2 shows whether the differences in average achievement among the countries are statistically 
significant. The top part of the exhibit shows the results for ePIRLS. Singapore was the top- 
performing country, followed by Norway and Ireland, which had higher achievement than all 
the other participating countries except Singapore. Sweden, Denmark, and the United States also 
performed very well on ePIRLS. 

For comparison purposes, the bottom part of the exhibit shows the PIRLS comparisons for the 
ePIRLS participants. In this case, Singapore was the top-performing country, followed by Ireland, 
Norway, Chinese Taipei, and Sweden. 
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Exhibit 1.2: Multiple Comparisons of Average Achievement 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Instructions: Read across the row for a country to compare performance with the countries listed 5 
along the top of the chart. The symbols indicate whether the average achievement of the country inthe rowis 4 
significantly lower than that of the comparison country, significantly higher than that of the comparison - 
country, or if there is no statistically significant difference between the average achievement of the > 
two countries. z 
2pome 
S 2 
2s 3 
oF fa 
y ta ea ze 5 
Country = 4 Ea?) eae 
o§ 98 ~ ae S2l25 
1% ce £5 2 uw « foe mem ¢ [> fea = 
an a? EBS fats fare B SPoagelsa = 
- wv as Yeas vs 29a S$] PA 
Su SStesececaecoe te sise 
ZARBZEARQADVIGLZZHReEVD alas s 
Singapore 588 (3.0) e©eoooooooooo0o 0 O|o/= 
Norway (5) 568 (2.2) © e©ooooooood°o ie 
Ireland 567 (2.5) © ©oooooo0o0od 0 o|/o/¢ 
Sweden 559 (23) © © @ ©0000 00 0 ©Oo5 
Denmark 558 (2.2) © © @ ©0000 000 00 
United States 557 (26) © © @ ©oooo0o00 0 (er 4) 
Chinese Taipei 546 (20) © © ®©@ ®@ @ o|o a/a|/a|0 oo 
Canada 543(3.2) MW @©@DW @OD ®D ®@ ©0000 00 
Israel 536 (23) OVD OHYDW GOW ®@D ® ooo 0 o°0 
Italy 532(21) ©BDGDWOADDHW®D ® 0000 (4) 
Slovenia 525(19 ®@®DWDWODW@DGPOA9@ GMAW GOA@GOD OGD ® 00 (4) © Average achievement significantly higher than 
Portugal 522(22) MDW DVDWPD@ OD D9 ® ® o°o ° comparison country 
Georgia 47733) © GB GD D9DDYDYDOD@® Y © (4) @ 0 @ Average achievement significantly lower than 
United Arab Emirates 468(22) PM MDM ODGPO@GPMW@ M9 AOD MW@ GOD ® ® 0 comparison country 
Benchmarking Participants 
Dubai, UAE 528(164) © VM YVOYWOW OW YW ®D ® 00 (4) 
Abu Dhabi, UAE = 431(4.1) © @ DW DDO PDW@ GOW DW PWDPD © ® ® 


PIRLS 


“ 
< 
c 
un 2 
ke 
et 
on 
Ree Ea a 
Country 2 foe fg eae) & 
— vu >~ 6 = om mw pw = 
25 ¥¢ Or & ~ te ley =F =< 2 
BA 80FREG SSE SSTDELS 
Sy ©f& SUDO Ean @zGu sa fa SIs 
2s SSSesteaesects se s(se 
ZAR BLZGOASLZASGZZL25E5S Blbe 
Singapore 576 (3.1) CoO C0 CC 0000000 4) 
Ireland 566 (2.8) © C©O0o00000 000 oOo 
Norway (5) 560 (2.3) © ©0000 0000 or) 
Chinese Taipei 559 (2.0) © C©Oo0000000 o°o 
Sweden 555 (2.4) © @ ©0000 0000 4) 
United States 550 (2.9) © @ ® @ oo0000 0° 
Italy 548 (24) ®© 9 @ @ ® 000 0 0° 
Denmark  548(23) @ @ © @ @ 0000 00 
Canada 543 (3.3) @ @ @ @ ® 0000 0° 
Slovenia 543 (2.0) © @ @ @ ®@ ® 0000 oo 
Israel 532 (25) OW DOW DW PDW@PDAADADA®GD ® (aa) © QO = © Average achievement significantly higher than 
Portugal 528(23) MW WDDMW@PD9 ADD GD ® 00 o°o comparison country 
Georgia 4893.1)© © DW DWDWDDDOA9@ GO YD © (4) ® Q @® Average achievement significantly lower than 
United Arab Emirates 451(27) © @DMBDWGPDW@A9GPDW@WGP@#WAGP@ AW GPD® ® ® oO comparison country 


Benchmarking Participants 


Dubai, UAE 516119) WV VDWOD@D YD YW GSW OTP OPW GPOO (4) 
Abu Dhabi, UAE = 414(482) © WW GWT HPPHWPDW DW YAW DPD ®Y ® ® 


Significance tests were not adjusted for multiple comparisons. Five percent of the comparisons would be statistically significant by chance alone. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 1.3: Average Achievement by Gender 


In ePIRLS 2016, fourth grade girls had higher average achievement than boys in all countries 
except Italy, Portugal, and Denmark, where achievement was similar for boys and girls. The average 
advantage for girls was 12 points across the 14 ePIRLS countries. Essentially, for these countries, the 
gender gap was similar to that on PIRLS (14 points). 
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Exhibit 1.3: Average Achievement by Gender 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Ke} 

ePIRLS Online 3 

(Absolute , = 

Students Scale Score Students Scale Score Scored Higher Scored Higher a 

Italy 49 (0.8) 534 (2.6) 0.8) 531 (2.4) 2 (2.6) a & 

2 Portugal 49 (0.8) 524 (2.6) 51 (0.8) 521 (2.6) 3 (2.7) = ct 

= Denmark 51 (0.9) 560 (2.9) 49 (0.9) 556 (2.9) 4 (3.8) fon] & 

t United States 50 (0.7) 560 (2.8) 50 (0.7) 554 (3.1) 6 (2.9) = é 

12 Canada 50 (1.0) 547 (3.7) 50 (1.0) 539 (3.7) 8 (3.8) free] = 

Chinese Taipei 48 (0.7) 551 (2:3) 52 (0.7) 541 (2.2) 9 (2.0) = 3 

Ireland 51 (1.7) 572 (2.8) 49 (1.7) 561 (3.4) 11 (3.6) [ence earaest 2 

3 Israel 51 (1.2) 542 (2.5) 49 (1.2) 530 (3.1) 11 (3.0) =e £ 

Slovenia 50 (0.8) 532 (2.5) 50 (0.8) 518 (2.5) 14 (3.3) [ay g 

Sweden 49 (1.1) 567 (2.6) 51 (1.1) 552 (2.7) 15 (2.5) el 3 

1 Georgia 49 (0.9) 485 (3.2) 51 (0.9) 469 (3.8) 15 (2.5) od 2 

Norway (5) 51 (1.2) 576 (2.6) 49 (1.2) 558 (2.9) 18 (3.2) ————| = 

3 Singapore 49 (0.6) 599 (3.2) 51 (0.6) 578 (3.3) 21 (2.8) coe eres E 

United Arab Emirates 49 (1.9) 483 (3.4) 51 (1.9) 454 (4.1) 29 (6.3) _ ee 7 
— 


Benchmarking Participants 


Dubai, UAE 48 (3.1) 534 (2.7) 52 (3.1) 522 (2.8) 12 (4.7) = 
Abu Dhabi, UAE 47 (3.5) 451 (6.6) 53 (3.5) 414 (6.5) 37 (10.6) , aE 
30 20 1 0 0 10 20 30 


(Absolute 


cer La [Ses | Sa | sn 
Country Value) / ‘ 
Students Scale Score Students Scale Score Scored Higher Scored Higher 
2 Portugal 49 (0.8) 529 (2.7) 51 (0.8) 527 (2.6) 1 (2.7) | 
Chinese Taipei 48 (0.7) 563 (2.2) 52 (0.7) 556 (2.3) 7 (1.9) =a 
Italy 49 (0.8) 552 (2.7) 51 (0.8) 545 (2.6) 7 (2.6) Feral 
t United States 50 (0.7) 555 (3.0) 50 (0.7) 545 (3.4) 9 (3.0) =z 
12 Canada 50 (1.0) 548 (3.6) 50 (1.0) 537 (3.7) 11 (3.4) aa 
Ireland 51 (1.7) 571 (3.3) 49 (1.7) 560 (3.6) 11 (4.1) Sa 
= Denmark 51 (0.9) 554 (3.1) 49 (0.9) 541 (2.8) 13 (3.6) a 
3 Israel 51 (1.2) 538 (2.8) 49 (1.2) 524 (3.4) 14 (3.7) Saaeeey 
Sweden 49 (1.1) 563 (2.6) 51 (1.1) 547 (2.8) 16 (2.5) a 
3 Singapore 49 (0.6) 585 (3.5) 51 (0.6) 568 (3.4) 17 (3.1) SS 
Slovenia 50 (0.8) 552 (2.3) 50 (0.8) 534 (2.7) 18 (3.1) es 
1 Georgia 49 (0.9) 499 (3.0) 51 (0.9) 480 (3.7) 19 (3.0) == em 
Norway (5) 51 (1.2) 571 (2.8) 49 (1.2) 549 (2.6) 22 (2.8) See 
United Arab Emirates 49 (1.9) 466 (4.0) 51 (1.9) 436 (4.6) 29 (6.8) ee 
ommnone 
Benchmarking Participants 
Dubai, UAE 48 (3.1) 523 (3.4) 52 (3.1) 510 (3.0) 13 (5.5) oT 
Abu Dhabi, UAE 47 (3.5) 435 (7.4) 53 (3.5) 395 (6.6) 40 (10.7) 
30 20 10 0 10 20 30 


@ Difference statistically significant 


™@! Difference not statistically significant 


See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 1.4: Average Achievement on ePIRLS Compared to PIRLS 

Informational Reading 

Exhibit 1.4 compares average achievement in ePIRLS informational online reading compared 
to only the PIRLS informational purpose for reading. The results are somewhat different than 
the comparison between ePIRLS and PIRLS overall (literary and informational reading). When 
comparing achievement in informational reading, the countries with higher achievement in ePIRLS 
included Denmark, the United States, Singapore, the United Arab Emirates, Norway, Israel, and 
Sweden. The countries with higher achievement in PIRLS informational reading included Portugal, 
Georgia, Italy, Slovenia, and Chinese Taipei. There was no difference in Ireland or Canada. 


Exhibit 1.4: Average Achievement on ePIRLS Compared to PIRLS 
Informational Reading 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Ko) 

Online s z 

Country Informational Informational Micacantes ePIRLS Online : a 

Average Scale i PIRLS Informational > 

Average Scale Informational Scare Maher 3 

Cane 108 Score is Higher 4 5 

= Denmark 558 (2.2) 544 (2.7) 15 (1.9) sa g 

t United States 557 (2.6) 543 (2.9) 13 (1.4) as 2 

3 Singapore 588 (3.0) 579 (3.3) 9 (1.1) a 3 

United Arab Emirates 468 (2.2) 460 (2.7) 8 (1.3) as] z 

Norway (5) 568 (2.2) 560 (2.4) 8 (1.5) = ES 

3 Israel 536 (2.3) 530 (2.4) 6 (1.4) = £ 

Sweden 559 (2.3) 555 (2.6) 5 (1.4) | z 

Ireland 567 (2.5) 564 (3.0) 3 (1.5) = % 

12 Canada 543 (3.2) 540 (3.2) 3 (1.4) a > 

2 Portugal 522 (2.2) 528 (2.4) -6 (1.1) = « 

1 Georgia 477 (3.3) 487 (3.4) -10 (2.4) ea S 

Italy 532 (2.1) 549 (2.4) -17 (1.9) SEE u 

Slovenia 525 (1.9) 544 (2.1) -19 (1.1) [aaa 3 

Chinese Taipei 546 (2.0) 569 (2.2) -24 (1.5) = 2 

Benchmarking Participants 
Abu Dhabi, UAE 431 (4.1) 422 (4.9) 9 (2.6) ran 
Dubai, UAE 528 (1.6) 524 (2.1) 4 (1.0) = 
30 20 10 0 10 20 30 


™ Difference statistically significant 


™ Difference not statistically significant 


See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 1.5: Achievement Differences Between ePIRLS and PIRLS Informational 
for Girls and for Boys 

Exhibit 1.5 shows graphs of average achievement on ePIRLS and PIRLS informational reading by 
gender, with the top part of the exhibit presenting the results for girls and the bottom part the 
results for boys. The countries are presented in alphabetical order in both parts. In nearly all of 
the countries, the results for the girls and for the boys mirror the national results. Both girls and 
boys had higher achievement on ePIRLS than PIRLS informational reading in Denmark, Israel, 
Norway, Singapore, United Arab Emirates, and the United States, and higher achievement on PIRLS 
informational reading in Chinese Taipei, Georgia, Italy, Portugal, and Slovenia. The results by gender 
were different in Ireland, where the national difference was not significant but girls had an advantage 
in ePIRLS, and in Sweden where the national results showed a difference favoring ePIRLS that was 
only significant for boys. 
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Grade 
Exhibit 1.5: Achievement Differences Between ePIRLS and PIRLS Informational PANTS cD 
for Girls and for Boys 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Ke} 

ePIRLS Online PIRLS y 

Informational Informational Difference ePIRLS lee PIRLS Informational & 

Country Average Scale Score} Average Scale Score Informational Score is High = 

aa igher s 

Score is Higher a 

12 Canada 547 (3.7 543 (3.6 = & 

Chinese Taipei 551 (233) 570 (2.7) -20 (2.3) a a 

= Denmark 560 (2.9 548 (3.5 12 (2.3 === = 

1 Georgia 485 (3.2) 496 (3.3) -11 (2.9) =a & 

Ireland 572 (2.8 568 (3.5 5 (1.7 = z 

3 Israel 542 (2.5) 534 (2.8) 7 (1.5) =z 3 

Italy 534 (2.6 551 (2.7 -18 (2.8 SSS eg g 

Norway (5) 576 (2.6) 570 (3.0) 7 (1.7) ay 'c 

2 Portugal 524 (2.6 527 (2.9 -4 (1.6 [ose § 

3 Singapore 599 (3.2) 586 (3.6) 13 (1.4) —— S 

Slovenia 532 (2.5 552 (2.6 -20 (2.3 [Ss a % 

Sweden 567 (2.6) 562 (3.1) 4 (2.2) = = 

United Arab Emirates 483 (3.4) 475 (4.0 8 (1.6 =e g 

t United States 560 (2.8) 547 (3.0) 12 (1.5) a 9 

545 (08) 005) 
Benchmarking Participants 
Abu Dhabi, UAE 451 (6.6) 443 (7.4) 8 (2.3) ay 
Dubai, UAE 534 (2.7) 530 (3.6) 4 (2.2) = 
30 20 10 0 10 20 30 


iit 


i i i PIRLS Onli 
Informational Informational Difference e nuine PIRLS Informational 


Country Average Scale Score| Average Scale Score re, Score is Higher 
Score is Higher 
12 Canada 539 (3.7 537 (3.8 : rt 

Chinese Taipei 541 (2.2) 568 (2.3) -27 (1.5) _———— 

= Denmark 556 (2.9 539 (3.0 17 (2.4 aT 

1 Georgia 469 (3.8) 478 (4.3) -9 (3.1) [as] 
Ireland 561 (3.4 560 (3.8 1(1.9 | 

3 Israel 530 (3.1) 526 (3.4) 523) = 
Italy 531 (2.4 547 (2.8 -16 (1.7 ES) 
Norway (5) 558 (2.9) 549 (2.9) 9 (2.7) [az 

2 Portugal 521 (2.6 529 (2.7 8 (17 == 

3 Singapore 578 (3.3) 572 (3.7) 6 (1.7) — 
Slovenia 518 (2.5, 537 (2.7 -19 (1.6 rs 
Sweden 552 (2.7) 547 (2.9) 5 (1.6) [razr] 
United Arab Emirates 454 (4.1 446 (4.8 8 (2.0 [| 

t United States 554 (3.1) 540 (3.6) 14 (1.9) as 

Benchmarking Participants 
Abu Dhabi, UAE 414 (6.5) 403 (7.3) 11 (4.2) ==) 
Dubai, UAE 522 (2.8) 519 (3.0) 3 (1.7) = 
30 20 10 0 10 20 30 


@ Difference statistically significant 


™@ Difference not statistically significant 


See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 1.6: Achievement in Comprehension Processes - ePIRLS Online 
Informational Reading 


PIRLS reports achievement according to two overarching comprehension processes: 


e Retrieving and straightforward inferencing 
e Interpreting, integrating, and evaluating 


Exhibit 1.6 shows the ePIRLS results for these two comprehension processes. Singapore, Chinese 
Taipei, Portugal, Georgia, and the United Arab Emirates had a relative advantage in retrieving 
and straightforward inferencing. Only the United States and Canada had a relative advantage in 
interpreting, integrating, and evaluating. The remaining countries essentially had no difference 
between the two processes. 


Exhibit 1.6: Achievement in Comprehension Processes - 
ePIRLS Online Informational Reading 
Note: Results based on students who participated in both PIRLS and ePIRLS. 


Ke) 
v Straightforward Inferencing and Evaluating z 
country Informational ; ; ; : a 
Average ene Difference None Difference rocess Score rocess Score s 
Scale Score Reiner from Overall eecre from Overall Lower than Overall Higher than Overall & 
ePIRLS Score ePIRLS Score ePIRLS Score ePIRLS Score 8 
3 Singapore 588 (3.0) 594 (3.3) 6 (0.7 (4) 585 (3.1 3(08) @ ae i 
Norway (5) 568 (2.2) 567 (2.2) 0 (1.4) 568 (2.3) 0 (1.1) 5 
Ireland 567 (2.5) 566 (2.4) -1 (0.9 568 (2.5, 1 (0.8) a & 
Sweden 559 (2.3) 561 (2.2) 1 (0.8) 559 (2.5) 0 (1.1) . 5 
= Denmark 558 (2.2) 560 (2.2) 2 (1.0 556 (2.6 -2 (13) on E 
t United States 557 (2.6) 553 (2.6) 3(08) @ 560 (2.6) 3(06) © a = 
Chinese Taipei 546 (2.0) 548 (2.1) 3 (0.6 (4) 544 (1.9 2 (08) @ en ra 
12 Canada 543 (3.2) 541 (3.0) 2(08) @ 545 (3.2) 2(08) © ~_ S 
3 Israel 536 (2.3) 536 (2.5) 0 (13 535 (2.4 -1 (1.0) rm 2 
Italy 532 (2.1) 534 (2.1) 2 (0.9) 531 (2.3) -2 (1.0) a 
Slovenia 525 (1.9) 525 (1.8) 0(1.1 523 (2.0 2(08) @ raat & 
2 Portugal 522 (2.2) 525 (2.4) 2(08) O 521 (2.1) 2(05) @ os a 
1 Georgia 477 (3.3) 485 (3.3) 8 (0.9 (4) 466 (3.7 -11(14) @ oe 
United Arab Emirates 468 (2.2) 471 (2.1) 2(0.6) Oo 465 (2.2) -3(0.4) @ ee 
Benchmarking Participants 
Dubai, UAE 528 (1.6) 528 (1.7) 0 (1.4) 527 (1.6) 0 (1.0) 
Abu Dhabi, UAE 431 (4.1) 434 (4.1) 3 (1.4) 428 (4.0) 3(0.9) @ —_ 
26 io H 16 20 
Subscale score significantly higher than overall ePIRLS score © @ Retrieving and Straightforward Inferencing 
Subscale score significantly lower than overall ePIRLS score @ Interpreting, Integrating, and Evaluating 


See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 1.7: ePIRLS Achievement in Comprehension Processes by Gender 
Similar to the national results, girls had higher achievement than boys in retrieving and 
straightforward inferencing in every country except Portugal. However, the advantage for girls 
occurred in fewer countries in interpreting, integrating, and evaluating. There was no gender gap 
for this higher order process in Canada, Denmark, Italy, Portugal, and the United States. 


Exhibit 1.7: Achievement in Comprehension Processes by Gender - 
ePIRLS Online Informational Reading 
Note: Results based on students who participated in both PIRLS and ePIRLS. 


ePIRLS Comprehension Processes 


Country Retrieving and Straightforwa' Interpreting, Integrating, and 
Inferencing Evaluating 


a 


<2. 

= 

2 

12 Canada 547 (3.5) O 5343.8 547 (3.7) 543 (3.8) 2 
Chinese Taipei 555 (2.7) @ 542 (2.2) 548 (2.3) © 540 (2.0) 2 
= Denmark 565 (3.0) © 555 (27 556 (3.3) 556 (3.0) 3 
1 Georgia 495 (3.3) @ 475 (4.0) 471 (3.7) © 461 (4.2) 3 
Ireland 572 (3.0) @ 559 (3.4) 573 (3.0) © 563 (3.4) £ 
3 Israel 544 (2.9) © 528 (3.0) 539 (2.5) @ 531 (3.4) iS 
Italy 537 (2.6) © 531 (2.4) 530 (2.6) 531 (2.8) 7 
Norway (5) 578 (2.8) @ 557 (2.9) 575 (2.8) @ 560 (2.9) 4 
2 Portugal 528 (2.7) 522 (3.0 521 (2.3) 521 (2.7) > 
3 Singapore 606 (3.6) © 583 (3.5) 594 (3.5) @ 575 (3.3) a 
Slovenia 535(2.1) © 516 (2.4) 528 (2.2) © 519 (2.6) a 
Sweden 570 (2.5) @ 551 (2.7) 565 (2.9) @ 553 (3.2) u 
United Arab Emirates 487 (3.4) 7 455 (4.1 478 (3.6) © 453 (4.1) 3 
t United States 558 (2.7) 548 (3.1) 562 (2.8) 557 (3.2) 7 


International Avg. 548 (0.8) 7 ESE (R:)) CY-V(\h:) en 4) em (1) 


Benchmarking Participants 


Abu Dhabi, UAE 456 (6.6) © 415 (6.6) 446 (6.6) © 412 (6.7) 
Dubai, UAE 535 (2.7) © 521 (3.0) 532 (2.8) 523 (2.8) 


© Average significantly higher than other gender 


See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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PAU 
ePIRLS Online Reading at the PIRLS International Benchmarks 


ePIRLS describes achievement at four International Benchmarks: 
Advanced, High, Intermediate, and Low 


ONLINE READING —- FOURTH GRADE 


Percentage of Students Reaching Benchmarks 
(averaged across countries) 
. 


Advanced \ 2 
ay - ----------- *}____i™! 1 2% 
(625) One-third of students 
reached the Advanced Benchmark 
in Singapore (34%) 


High 
Bonchinark 222 cee Se eect cee ee 5 Q% a 2 


(550) 


Intermediate \* 2 
Benchmark 84% 


C1 pened 


Yi 
Benchmark - - - - 7 a 
a ina © si 


In ePIRLS, 
Students Demonstrated 
Impressive Competence in Online Reading 


Half the students reached the High International Benchmark (50% across countries) 


Students reaching this benchmark can: 
+ Integrate information across webpages with interactive features 
« Evaluate how graphic elements support content 


Almost all students reached the Low International Benchmark (97% across countries) 
Students reaching this benchmark can: 
- Locate information on webpages with a variety of dynamic 
and navigable features 


TIMSS & PIRLS 


SOURCE: IEA's Progress in International Reading L, incernationalstud 
4) ly Center 
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CHAPTER 2 


Performance at International 
Benchmarks 


ePIRLS Benchmarking at the PIRLS International Benchmarks 


To provide an interpretation of the results summarized on the PIRLS achievement scale for reading 
comprehension at the fourth grade, PIRLS describes achievement at four points along the scale 
as international benchmarks: Advanced International Benchmark (625), High International 
Benchmark (550), Intermediate International Benchmark (475), and Low International Benchmark 
(400). To develop the descriptions, the TIMSS & PIRLS International Study Center conducted a 
scale anchoring analysis together with the PIRLS 2016 Reading Development Group (RDG). The 
descriptions of achievement at the International Benchmarks are based on the reading skills and 
strategies demonstrated by fourth grade students achieving at each level of the scale, respectively. 
Further detail about the scale anchoring methodology is provided in Chapter 13 of Methods and 
Procedures in PIRLS 2016. 

For PIRLS 2016, the scale anchoring analysis was conducted for ePIRLS as well as PIRLS. This 
enabled reporting descriptions of ePIRLS achievement at the Advanced, High, Intermediate, and 
Low Benchmarks. 


Overview of the ePIRLS 2016 Tasks and Items 


The ePIRLS tasks and items used in 2016 were selected and developed based on the PIRLS 2016 
Assessment Framework. The framework describes the PIRLS view of reading as an interactive process 
between the text and the reader and describes the ways that PIRLS measures students’ reading 
achievement. It specifies that the assessment texts and items should cover in equal amounts the two 
purposes that account for most of the reading done by young students in and out of school: literary 
and informational. The ePIRLS tasks assess reading for informational purposes, but on the Internet 
in an environment of interconnected webpages and a variety of visual and textual elements. 

Just like PIRLS, the items accompanying the ePIRLS informational tasks measure four 
processes of comprehension: retrieving, straightforward inferencing, interpreting and integrating, 
and evaluating and critiquing. These are collapsed into two processes for reporting: retrieving and 
straightforward inferencing and interpreting, integrating, and evaluating. 
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Description of the ePIRLS Tasks Assessing Online Informational Reading 

There were five ePIRLS tasks simulating online informational reading similar to what might be 
required for school projects and reports. They were about science or social studies topics, including: 
“Mars, “Elizabeth Blackwell (the first woman doctor),” “Rainforests,” “Troy,” and “Animal Migration” 
Two of the tasks—“Mars” and “Elizabeth Blackwell”—are released to provide examples of the ePIRLS 
tasks. To see the released examples, watch the ePIRLS 2016 Example Tasks video or Take the ePIRLS 
Assessment. Taking the ePIRLS test includes both released tasks in their entirety as they were given 


to students. You can type in your answers, and the scoring guides are provided at the end so you 
can check your answers. 
An overview of the ePIRLS tasks is provided below. 


ePIRLS 


The five online informational tasks each offered a simulated Internet environment on an informational topic, 
including scientific and historical subject matter. Each task was structured as a class project or report, with 
an avatar teacher who introduced the questions and guided the student through the task. The tasks were 
conceptually relatively demanding, some of them based on abstract or technical ideas and with a 
substantial number of embedded details. Each task involved students working across approximately three 
different websites totaling about five to ten webpages with an average of 1,000 words of text per task. In 
addition to the text, the tasks included different kinds of visual information such as photos, charts, and maps 
as well as many navigational and dynamic features such as animations, hyperlinks, tabs, and pop-up boxes. 
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Description of ePIRLS Achievement at the PIRLS International Benchmarks 


The graphic shows the descriptions of ePIRLS achievement demonstrated by fourth grade students 
at each of the four International Benchmarks. 


ee Advanced International Benchmark 


6) 2 5 When reading and viewing relatively complex Online Informational Texts, students can: 


+ Make inferences from complex information to support an explanation 


+ Interpret and integrate information from within and across webpages with interactive features to 
explain relationships, and show thorough understanding 


+ Evaluate the effects of textual, visual, and interactive elements and begin to consider the writer's 
point of view 


Ce High International Benchmark 


5 5 @) When reading and viewing relatively complex Online Informational Texts, students can: 


* Make inferences to distinguish relevant information and provide comparisons 


+ Interpret and integrate information within and across webpages with interactive features to 
provide examples and make contrasts 


* Evaluate how graphic elements and language choices support content 


@e Intermediate International Benchmark 


4 7, 5 When reading and viewing relatively complex Online Informational Texts, students can: 


+ Locate and reproduce information presented in various forms including independent use of 
navigation features 


* Make straightforward inferences to recognize reasons and actions 


+ Interpret and integrate information across a webpage to recognize causes, comparisons, and 
explanations 


* Begin to evaluate the use of interactive features to convey information 


{ Jae Low International Benchmark 


rALOl@) When reading and viewing relatively complex Online Informational Texts, students can: 


+ Locate and reproduce explicitly stated information from webpages that contain text and a 
variety of dynamic, navigable features (e.g., timelines, Pop-up boxes) 


* Begin to make straightforward inferences about descriptions 
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Exhibit 2.1: ePIRLS Online Informational Reading Performance at the PIRLS 
International Benchmarks 

Exhibit 2.1 presents the percentage of ePIRLS students reaching each International Benchmark. 
The results are presented in descending order according to the percentage of students reaching the 
Advanced International Benchmark. The percentage of students reaching the Advanced Benchmark 
is indicated in the bar graph with a black dot. Because students who reached the Advanced 
Benchmark also reached the other benchmarks, the percentages illustrated in the exhibit and shown 
in the columns to the right are cumulative. About one-third of the fourth grade students reached 
the Advanced International Benchmark in Singapore (34%), with the next highest percentage in 
Ireland (20%). 

As a point of reference, Exhibit 2.1 provides the median percentage of students reaching each 
benchmark at the bottom of the four right-hand columns. By definition, half the countries will 
have a percentage in that column above the median and half will be below the median. The median 
percentages of students reaching the International Benchmarks were as follows: Advanced—12 
percent, High—50 percent, Intermediate—84 percent, and Low—97 percent. Most of the ePIRLS 
countries (12) had 93 percent or more of their students reaching the Low Benchmark. 


Exhibit 2.1: ePIRLS Online Informational Reading Performance at the 

PIRLS International Benchmarks 

Note: Results based on students who participated in both PIRLS and ePIRLS. 

@ Advanced 

Percentages of Students Reaching O High 
International Benchmarks © Intermediate 

© Low 


Advanced High Intermediate Low 


Benchmark Benchmark Benchmark Benchmark 
(V2) (6371) (CY) (400) 


Country 


© 
8 
= 
3 Singapore eo 34(17) 72 (1.5) 92 (0.9) 98104) 
Ireland @—o 20 (1.3) 63 (1.6) 90 (0.9) 98 (0.4) 5 
Norway (5) e Oo @—o 18 (1.2) 63 (1.6) 92 (0.8) 99 (0.3 = 
t United States @ ie) @ oO 18 (1.2) 56 (1.5) 86 (1.0) 97 (0.4) 5 
= Denmark e Oo @ oO 15 (1.0) 57 (1.4) 89 (0.9) 98 (0.4 a 
Sweden e ° e—o  14(10) 59 (1.7) 89 (0.8) 9803) § 
3 Israel e oO @ Oo 13 (1.0) 47 (1.3) 78 (1.1) 93 (0.7 7 
12 Canada e Oo e ro) 12 (1.0) 49 (2.1) 82 (1.3) 9605) # 
Chinese Taipei e oO e oO 10 (0.7) 51 (1.3) 86 (0.9) 97 (0.3 £ 
Italy e roy e ° 6 (0.7) 41 (13) 82 (1.3) 98 (0.5) 
Slovenia e re) e ° 5 (0.5) 39 (1.3) 78 (1.0) 95 (0.6 2 
United Arab Emirates e O e oO 5 (0.3) 22 (0.8) 50 (1.1) 75 (0.9) = 
2 Portugal e O e ° 5 (0.6) 35 (1.4) 77 (13) 97 (0.5 ei 
1 Georgia e O e ro) 1 (0.4) 16 (1.3) 54 (2.1) 8514) 
° 6 ° 0 5 
Benchmarking Participants 
Dubai, UAE e oO @ O 12 (0.5) 44 (0.9) 75 (0.8) 91 (0.4) 
Abu Dhabi, UAE -@ O oO O 3 (0.5) 13 (1.2) 36 (1.8) 61 (1.7) 
i T T T 1 
0 25 50 75 100 


See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.2: Low International Benchmark (400) 


Exhibit 2.2 presents the description of ePIRLS students’ achievement at the Low International 
Benchmark. Essentially, students could locate and reproduce information from webpages with a 
variety of dynamic and navigable features. 

Exhibits 2.2.1 through 2.2.3 contain three example items. Each exhibit shows the webpage the 
item is based on together with the item. The exhibit shows the achievement results for the countries 
that participated in ePIRLS, with up and down arrows indicating a significantly higher or lower 
percentage of success than the international average. The reading comprehension process and scale 
anchoring description are provided above the item. For multiple-choice items, the correct response 
is indicated. Constructed response questions were worth 1, 2, or 3 points. Each constructed response 
item is shown with an illustrative student response and the amount of credit awarded the response 
is shown across the bottom of the exhibit, usually full credit. 

Example Items 2.2.1 (multiple-choice) and 2.2.2 (constructed response) from the “Mars” task 
about space exploration show that students at the Low International Benchmark demonstrated 
that they could retrieve explicitly stated details from text and from a pop-up window. Students also 
were able to make an inference to select the correct search result to learn about Doctor Elizabeth 
Blackwell (Example Item 2.2.3). 


Exhibit 2.2: ePIRLS 2016 Online Informational Reading at the 
Low International Benchmark (400) 


400 When reading and viewing relatively complex Online Informational Texts, students can: 


* Locate and reproduce explicitly stated information from webpages that contain text and a 
variety of dynamic, navigable features (e.g., timelines, pop-up boxes) 


* Begin to make straightforward inferences about descriptions 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 
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Exhibit 2.2.1: Low International Benchmark for ePIRLS Online Informational Reading - 
Example Item 1 


Process: Focus on and Retrieve Explicitly Stated Information 
Description: Locate and recognize an explicitly stated reason 


ePIRLS Online Reading 2016 
http://www.eightplanets.org/mars j PIRLS 


[ine Soar System 7 Class Project 
ee ey ‘8 PLANETS": 


‘| Home | Sun | Mercury | Venus | Earth [Mars Jupiter | Satu | Uranus 


The Red 


Planet 


For hundreds of years, scientists have been 
interested in Mars. Because it is a planet 
next to Earth, perhaps it also has life? 


At first, scientists could only study Mars 
through telescopes. They could see that Mars 
looked red. They found out that Mars is 
about half the size of Earth. Interestingly, 
until about 100 years ago, scientists thought 
that Mars had man-made canals. Because all 


thought there might be life on Mars! 


As technology improved, we learned more 
about Mars. Now we know that Mars’ color 
comes from reddish iron oxide on its surface. 
We learned that it is like a desert—very rocky 
and dusty. Also, Mars is farther away from 
the Sun than Earth, so it is much colder. In 
the winter, the temperature on Mars is 
about minus 100 degrees Celsius. 


5. 


Why is Mars colder than Earth? 


i, ; Student 


It is farther from the Sun. 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


living things on Earth need water, scientists 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


country Percent 
Correct 

Norway (5) 94 (0.6) © 

Sweden 93 (0.8) © 

3 Singapore 93 (0.7) O 
Ireland 90 (1.1) © 

t United States 89 (1.1) O 

= Denmark 89 (1.1) 
Slovenia 88 (0.9 

12 Canada 88 (1.2) 

3 Israel 87 (0.9 

A) 

2 Portugal 85 (1.0 
Chinese Taipei 85 (1.0) @ 
Italy 83 (1.0) @ 

1 Georgia 80 (1.5) @ 
United Arab Emirates 76 (0.9) @ 

Benchmarking Participants 
Dubai, UAE 86 (0.9 
Abu Dhabi, UAE 69 (1.6) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 
See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.2.2: Low International Benchmark for ePIRLS Online Informational Reading - 
Example Item 2 


Process: Focus on and Retrieve Explicitly Stated Information 


Description: Retrieve and reproduce the definition of a term from a pop-up text box 


ePIRLS Online Reading 2016 : 
http:/www.mars-exploration-program.org/gettingtomars PIRLS 


Class Project 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


HAVE A 
STAR 


anes are farther apart So, to get 
‘Mars’ orbit. Then, you must aim for ‘ 
when your rocket gets there. It will take your 2 4 NAMED 


months to get to Mars. AFTER 
Close [X] YOU OR 


KA 
" FRIEND! 


9: 


According to the website, what is 
an orbit? 


ie Ey Student 


A path around a star, 


; 


planet, or moon 


The answer shown illustrates the type of student response that would receive full credit (1 point). 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Country Percent 
Full Credit 
3 Singapore 74 (1.1) © 
Norway (5) 70 (1.5) © 
Italy 68 (1.5) O 
12 Canada 68 (1.5) © 
lreland 64 (1.9) © 
t United States 63 (1.5) O 
= Denmark 62 (1.9) © 
2 Portugal 56 (1.7) 
Chinese Taipei 55 (1.3) 
Slovenia 53 (1.6) @ 
Sweden 49 (18) @ 
3 Israel 48 (13) @ 
United Arab Emirates 42 (0.9) @ 
1 Georgia 28 (1.9) @ 


Benchmarking Participants 
Dubai, UAE 56 (0.9) 
Abu Dhabi, UAE 36 (1.7) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 
See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.2.3: Low International Benchmark for ePIRLS Online Informational Reading - 
Example Item 3 


Process: Make Straightforward Inferences 


Description: Make a straightforward inference a list of Internet search results to recognize ost relevant website 


http://www.google.com?doctor+elizabeth+blackwell PIRLS 


Class Project 
Google Doctor Elizabeth Blackwell gu ES 


Elizabeth J. Blackwell — Film Archive 

www.filmarchive.com/eblackwell_(film) 

Elizabeth Blackwell, (born Nov 2, 2002) is a child actress who is famous for her appearance in the series 
of movies, “Magic Mischief"... 


% Doctor Elizabeth Blackwell - Her Story 
www.womenshistory.org/dreblackwell 
Doctor Elizabeth Blackwell, (3 February 1821 — 31 May 1910) is recognized as the first woman to 
become a doctor in the United States of America... 


Elizabeth Blackwell Medal 
www.elizabethblackwellmedal.org 1. 
The Elizabeth Blackwell Medal is an award given to a doctor who shows exceptional... Look at the Google search results, 


Doctor Blackwell visits the jungle - Blossom Books at left. 
www.blossonmbooks/blackwell.com “4 Student 


Doctor Blackwell visits the jungle is a children’s picture book by Sarah Schubert. Published in 2010... } 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


Click on the link that is most likely 
to have information about the life 
and achievements of Doctor 
Elizabeth Blackwell. 


* Correct answer 


Google and the Google logo are registered trademarks of Google Inc., used with permission. 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Percent 
Country 
(Qitsay 
Sweden 93 (0.9) O 
= Denmark 93 (1.1) © 
Norway (5) 88 (1.0) © 
2 Portugal 87 (0.9) © 
Italy 87 (0.9) O 
lreland 83 (1.4) 
Chinese Taipei 83 (1.1 
(03) 
12 Canada 81 (1.4 
Slovenia 79 (13) @ 
1 Georgia 78 (1.5) @ 
t United States 78 (1.1) @ 
3 Israel 76 (1.2) @® 
3 Singapore B (0.9) @ 
United Arab Emirates 65 (0.8) @ 
Benchmarking Participants 
Dubai, UAE B (1.1) @ 
Abu Dhabi, UAE 62 (1.7) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 
See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.3: Intermediate International Benchmark (475) 


Exhibit 2.3 presents the description of ePIRLS achievement at the Intermediate Benchmark. Because 
the scale anchoring descriptions are cumulative, with students’ comprehension processes building 
on skills demonstrated at the lower levels, as anticipated students at the Intermediate Benchmark 
demonstrated greater facility in locating and reproducing explicitly stated information as well as 
skills in making inferences, interpreting and integrating information across webpages, and beginning 
to evaluate interactive features. 

Exhibits 2.3.1 through 2.3.3 present three example items. Each exhibit shows achievement 
results, with up and down arrows indicating a significantly higher or lower percentage of success for 
the country compared to the international average on the item. The reading comprehension process 
and scale anchoring description are provided above the item. For multiple-choice items, the correct 
response is indicated. Constructed response questions were worth 1, 2, or 3 points. Each constructed 
response item is shown with an illustrative student response and the amount of credit awarded the 
response is shown across the bottom of the exhibit, usually full credit. 

Example Item 2.3.1 shows that students were able to locate information about Elizabeth 
Blackwell by scrolling through a timeline and 2.3.2 shows they could provide a reason from the 
text—both were constructed response questions. Example 2.3.3 was one of the most difficult items 
in the ePIRLS assessment, based on drawing an inference from text and an animation showing the 
orbits of Earth and Mars around the Sun. Even students at the Advanced Benchmark did not provide 
a complete answer. However, it is interesting that readers at the Intermediate Benchmark understood 
some part of the difficulty in planning to get a rocket from Earth to Mars. 


Exhibit 2.3: ePIRLS 2016 Online Informational Reading at the 
Intermediate International Benchmark (475) 


@ Intermediate 


47 S) When reading and viewing relatively complex Online Informational Texts, students can: 


+ Locate and reproduce information presented in various forms including independent use of 
navigation features 


* Make straightforward inferences to recognize reasons and actions 


* Interpret and integrate information across a webpage to recognize causes, comparisons, and 
explanations 


* Begin to evaluate the use of interactive features to convey information 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 
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Exhibit 2.3.1: Intermediate International Benchmark for ePIRLS Online 
Informational Reading - Example Item 1 


Class Project 


Timeline 1821-1910 


@  Born3 February in Bristol, England, Elizabeth is the third 
of nine children. 


Elizabeth's family moves to New York, America. 


Elizabeth is accepted into Geneva Medical College, New 


York to study medicine. 


SOURCE: IEA's Progress in International Reading Literacy Study - PIRLS 2016 


Elizabeth finishes university at the top of her class and is 
the first woman ever to receive a medical degree in 
America. Elizabeth travels to France where she works at 
La Maternité Hospital. 


Elizabeth travels back to America and opens up her own | 
clinic, The New York Infirmary. 4. 

According to the timeline, what did 
Elizabeth trains nurses who will treat injured soldiers Elizabeth do in 1874? 


fighting in the Civil War in the United States. Py} Student 


Elizabeth returns to England and opens a clinic in 
London, becoming the first female doctor in England. She started a medical 


Elizabeth helps to create the London School of Medicine 
for Women. 


The answer shown illustrates the type of student response that would receive full credit (1 point). 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Country Percent 
Full Credit 
2 Portugal 79 (13) © 
Chinese Taipei 76 (1.2) © 
Sweden 75 (1.6) © 
3 Singapore 75 (0.9) © 
Italy B (15) © 
Ireland 72 (1.6) 
3 Israel 71 (1.2) 
= Denmark 70 (1.8) 
a) 
Norway (5) 66 (1.7) 
Slovenia 65 (1.8) @ 
12 Canada 65 (1.5) @ 
Tt United States 63 (16) @ 
United Arab Emirates 60 (0.9) @ 
1 Georgia 59 (1.9) @ 
Benchmarking Participants 
Dubai, UAE 69 (1.0 
Abu Dhabi, UAE 53 (18) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 


See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.3.2: Intermediate International Benchmark for ePIRLS Online 
Informational Reading - Example Item 2 


Process: Make Straightforward Inferences 
Description: Make a straightforward inference to provide a reason 


PIRLS 
Class Project 


Elizabeth's Goal Practicing 


A LADY DOCTOR? 
You Must Be Joking! 


SOURCE: IEA's Progress in International Reading Literacy Study - PIRLS 2016 


Practicing 
Within a few years of Elizabeth leaving college, more than 20 women throughout the 
country were studying in medical schools, including Elizabeth’s younger sister, Emily. 


America, however, was still not ready for a “lady doctor.” No hospital in America 
would hire Elizabeth, so she travelled to France to work in a hospital there. Elizabeth 
worked hard and started studying to become a surgeon. However, she became ill month with 
and lost sight in one eye and had to give up her dream of becoming a surgeon. i Roverphone 9. 
Despite this, she kept working hard as a general doctor. i Why did Elizabeth have to give up 
: her dream of becoming a surgeon? 
y Student 
! 


Unlimited free 
calls for the first 


She lost sight in one of 
her eyes. 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Percent 


Count 
Sania. Full Credit 


3 Singapore 84 (1.1) @ 
Norway (5) 80 (1.4) © 
= Denmark 79 (1.5) © 
Sweden 77 (1.7) © 
lreland 76 (1.7) @ 
Chinese Taipei B (1.2) O 
12 Canada 69 (1.3) 
t United States 67 (1.7) 
3 Israel 65 (1.4) 
Slovenia 61 (13) @ 
1 Georgia 60 (1.8) @ 
2 Portugal 59 (1.5) @ 
Italy 47 (16) @ 
United Arab Emirates 46 (0.9) @ 
Benchmarking Participants 
Dubai, UAE 64 (15) @ 
Abu Dhabi, UAE 35 (17) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 
See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.3.3: Intermediate International Benchmark for ePIRLS Online 
Informational Reading - Example Item 3 


Process: Interpret and Integrate Ideas and Information 


Description: Interpret complex information in text and an animated graphic to provide a partial explanation 


ePIRLS Online Reading 2016 ; 
http:/www.mars-exploration-program.org/gettingtomars PIRLS 


ihe Solansystems/” Mars Exploration Program ; Class Project 


id 
Rover Called Curiosity 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


HAVE A y 
STAR 


NAMED 

AFTER 

Close x] YOU OR 
XA 

" FRIEND! 


11. 

You have to plan a long time 
ahead to get to Mars. 
Explain why. 


) Student 


It takes a long time to 
get to Mars. 


The answer shown illustrates the type of student response that would receive partial credit (1 of 2 points). 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Percent At 


Countr 
y Least 1 Point 


3 Singapore 83 (0.8) © 
Norway (5) 64 (1.4) © 
Chinese Taipei 64 (1.5) © 

12 Canada 62 (1.6) © 
Ireland 60 (1.8) 

3 Israel 60 (1.4) 
Sweden 58 (1.5) 

t United States 58 (1.5) 
Italy 58 (1.4) 

51 (08) 

= Denmark 57 (1.9) 
Slovenia 48 (1.6) @ 
United Arab Emirates 44 (0.8) @ 

1 Georgia 44 (1.8) @ 

2 Portugal 40 (13) @ 

Benchmarking Participants 
Dubai, UAE 59 (1.1) 
Abu Dhabi, UAE 35 (13) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 
See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.4: High International Benchmark (550) 


Exhibit 2.4 contains the description of comprehension skills and strategies demonstrated by ePIRLS 
students at the High International Benchmark. At the High International Benchmark, students 
demonstrated that they could distinguish relevant information to provide comparisons; interpret 
and integrate information across webpages to make contrasts; and evaluate how graphic elements 
and language choices support content. 

Exhibits 2.4.1 through 2.4.4 contain examples of the types of items successfully answered by 
students achieving at the High International Benchmark. Each exhibit shows achievement results 
for the countries that participated in ePIRLS, with up and down arrows indicating a significantly 
higher or lower percentage of success than the international average. The reading comprehension 
process and scale anchoring description are provided above the item. For multiple-choice items, 
the correct response is indicated. Constructed response questions were worth 1, 2, or 3 points. Each 
constructed response item is shown with an illustrative student response and the amount of credit 
awarded the response is shown across the bottom of the exhibit, usually full credit. 

Example Item 2.4.1 illustrates that students were able to make an inference to recognize a 
definition presented via text and images. In Example Item 2.4.2 they could provide either a positive 
or negative reason to live in New York in the 1850s. Example 2.4.3 is a complex example, where 
students demonstrated that they were able to navigate across four sets of images and text to describe 
the capabilities of different parts of a Mars rover. In Example 2.4.4, they evaluated the purpose of 
the animated diagram showing Earth and Mars orbiting around the Sun. 


Exhibit 2.4: ePIRLS 2016 Online Informational Reading at the 
High International Benchmark (550) 


5 5 @) When reading and viewing relatively complex Online Informational Texts, students can: 


* Make inferences to distinguish relevant information and provide comparisons 


+ Interpret and integrate information within and across webpages with interactive features to 
provide examples and make contrasts 


* Evaluate how graphic elements and language choices support content 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 
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Exhibit 2.4.1: High International Benchmark for ePIRLS Online Informational Reading - 
Example Item 1 


Process: Make Straightforward Inferences 


Description: Make a straightforward inference to recognize a definition from text and images 


ePIRLS Online Reading 2016 a 
http://www.mars-exploration-program.org/missions PIRLS 


The Sola Sysiemy/_Vars Exporation Program 7 a Class Project 


Rover Called Curiosity 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


12. 
Which describes a flyby mission? 


’ Student 
-Orbiters te 


‘By the year 2000, countries were able to put spacecraft into orbit i 
around Mars. Long-term studies were now possible. Today, ‘ Spacecraft is flown by a human 
several spacecrafts are still orbiting Mars. 


Spacecraft takes pictures as it 
goes by Mars. 

Rovers 

In recent years, scientists thought of ways to put rovers on Mars. 
A rover is a remote-controlled vehicle with six wheels. It is the size 
of a small car. It can travel around and explore the surface of 
Mars. 


Spacecraft is flown by remote- 
control. 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Percent 
couney Correct 
lreland 77 (1.5) © 

= Denmark 76 (1.6) © 
t United States 75 (1.2) © 
3 Singapore 75 (1.0) © 
Sweden 75 (1.5) © 
Norway (5) 23 (1.4) © 
12 Canada 71 (16) © 
3 Israel 70 (1.3) © 
Chinese Taipei 68 (1.1) 
Italy 62 (13) @ 
Slovenia 61 (1.4) @ 
2 Portugal 60 (1.4) @ 
United Arab Emirates 56 (0.8) @ 
1 Georgia 42 (2.1) @ 


Benchmarking Participants 
Dubai, UAE 71 (1.1) © 
Abu Dhabi, UAE 49 (1.7) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 
See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t. +. and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.4.2: High International Benchmark for ePIRLS Online Informational Reading - 
Example Item 2 


CoM Tahecvepe-iccm(e(--eer-laveMalcoautd(eya) 


Description: Interpret and integrate information to draw a conclusion and support it with evidence 


PIRLS 


ony/ The New Yor nirmary Ji) js OE: C13 6 Coy (Lei F 


Need for Doctors Need for Patients A New Hospital 13. 
According to the webpage, would 
New York have been a good place 


Need for Doctors Elizabeth Blackwell to live in the 1850s? 


New York in the 1850s was already a huge city. Student 


People from all over the world were pouring in, Opens The New 
hoping to start a new life. Overcrowding and 

lack of money meant that deadly diseases York Infirmary 
spread quickly and no matter how many 

doctors there were, New York always needed 

more. 


Use information from the text to 
explain your answer. 


You could start a new life 


The answer shown illustrates the type of student response that would receive full credit (1 point). 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Country Percent 
Full Credit 
Ireland 78 (1.4) © 
Sweden 75 (1.4) © 
Norway (5) 75 (1.6) © 
= Denmark 74 (1.5) © 
3 Singapore 74 (1.3) © 
t United States 71 (1.6) © 
12 Canada 70 (1.7) © 
a) 
Slovenia 59 (1.2 
3 Israel 58 (13) @ 
Italy 57 (1.6) @ 
2 Portugal 52 (13) @ 
Chinese Taipei 41 (1.4) @ 
United Arab Emirates 36 (0.7) @ 
1 Georgia 35 (1.7) @ 
Benchmarking Participants 
Dubai, UAE 57 (0.7) @ 
Abu Dhabi, UAE 27 (13) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 


See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.4.3: High International Benchmark for ePIRLS Online Informational Reading - 
Example Item 3 


Process: Interpret and Integrate Ideas and Information 


terpret and integrate textual and visual information from a w age to recognize 4 functions by navigating across interactive images 


ePIRLS Online Reading 2016 = 
http://www.mars-exploration-program.org/rover-called-curiosity PIRLS 


Class Project 


16. 

WH EELS and LEGS ’ Match each part of Curiosity with 
something that the part does. 

Click on the drop-down menus. 


Pe) Student 


A. Arm and Hand 
collect rocks 


B. Body and Instruments 
analyze rocks 


take pictures 


maintain balance 


The answer shown illustrates the type of student response that would receive full credit (2 points). 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Country Percent 
Full Credit 
3 Singapore 70 (1.3) © 
Chinese Taipei 70 (1.2) © 
Norway (5) 54 (2.0) © 
t United States 54 (16) O 
Ireland 53 (2.1) O 
Sweden 50 (1.4) O 
= Denmark 48 (1.8 
12 Canada 48 (2.0) 
Italy 47 (1.6 
7 (08) 
3 Israel 39 (14) @ 
Slovenia 37 (16) @ 
2 Portugal 35 (16) @ 
United Arab Emirates 29 (0.9) @ 
1 Georgia 16 (1.3) @ 
Benchmarking Participants —— 
ey Dubai, UAE 49 (1.0 
Abu Dhabi, UAE 21 (13) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 


See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.4.4: High International Benchmark for ePIRLS Online Informational Reading - 
Example Item 4 


Process: Evaluate and Critique Content and Textual Elements 
Description: Evaluate the use of an animated diagram to determine its purpose 


ePIRLS Online Reading 2016 
http://www.mars-exploration-program_org/gettingtomars PIRLS 


LL?’ a Class Project 


il 


Rover Called Curiosity 


10. 
Look at the moving diagram of 
Earth and Mars orbiting around 
pl. are the Sun. What is the main purpose 
apart. So, to get F pes of the diagram? 
en, you must aim for EF Ae Student 
ere. It will take your : NAMED 
i ie : AFTER 
Close [X] b'40) 080) :1 
KA to show the colors of Earth and 
" FRIEND! 


to show that the Sun is part of 
the solar system 


to show that the distance 
between Earth and Mars changes 


to show that Earth and Mars are 
far from the Sun 


BeA 
Star! 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Percent 


Country core 


Norway (5) 85 (1.0) © 
Sweden 85 (1.4) © 
= Denmark 84 (1.3) © 
lreland 78 (1.5) © 
3 Singapore 77 (1.1) © 
Slovenia 75 (1.5) © 
t United States 75 (1.1) © 
12 Canada 75 (1.3) 
Chinese Taipei 70 (1.3) 
2 Portugal 70 (1.3) @ 
3 Israel 67 (1.2) @ 
Italy 66 (16) @ 
United Arab Emirates 52 (1.1) @ 


1 Georgia 50 (1.8) @ 


Benchma king | 


Abu Dhabi, UAE 44 (2.1) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 
See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.5: Advanced International Benchmark (625) 


Exhibit 2.5 describes the reading comprehension skills and strategies demonstrated by fourth 
grade students at the Advanced International Benchmark. Students at the Advanced International 
Benchmark could make inferences from complex information; interpret and integrate information 
within and across webpages to show thorough understanding; and evaluate the effects of features 
to begin to understand the writer’s point of view. 

Exhibits 2.5.1 through 2.5.6 contain six examples to demonstrate the range in the types of items 
successfully answered by students achieving at the Advanced International Benchmark. Each exhibit 
shows achievement results for the countries that participated in ePIRLS, with up and down arrows 
indicating a significantly higher or lower percentage of success than the international average. The 
reading comprehension process and scale anchoring description are provided above the item. For 
multiple-choice items, the correct response is indicated. Constructed response questions were worth 
1, 2, or 3 points. Each constructed response item is shown with an illustrative student response and 
the amount of credit awarded the response is shown across the bottom of the exhibit, usually full 
credit. 

Exhibits 2.5.1 and 2.5.2 show examples of students making inferences from relatively complex 
text to answer constructed response questions. Example Items 2.5.3 and 2.5.4 illustrate how students 
at the Advanced International Benchmark were able to interpret and integrate information across 
websites, Example 2.5.5 shows they could evaluate text to explain a writer’s point of view, and 
Example 2.5.6 shows that they could evaluate the use of a timeline. 


Exhibit 2.5: ePIRLS 2016 Online Informational Reading at the 
Advanced International Benchmark (625) 


@ Advanced 


62 s) When reading and viewing relatively complex Online Informational Texts, students can: 


* Make inferences from complex information to support an explanation 
+ Interpret and integrate information from within and across webpages with interactive features to 
explain relationships, and show thorough understanding 


* Evaluate the effects of textual, visual, and interactive elements and begin to consider the writer's 
point of view 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 
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Exhibit 2.5.1: Advanced International Benchmark for ePIRLS Online 
Informational Reading - Example Item 1 


Process: Make Straightforward Inferences 


PIRLS 
Class Project 


The Gift of Curiosity 


By Maria Green 


For decades, scientists have wondered whether the 
surface of Mars ever had water. Now, Curiosity’s 
cameras show signs that Mars once had rivers. There 
are photographs of what looks like a dried riverbed. 
Other photographs show huge canyons and valleys that 
could have been made by rivers. 


SOURCE: IEA's Progress in International Reading Literacy Study - PIRLS 2016 


Scientists are now “convinced that Mars did have an 
ocean a few billion years ago,” says Charles Elachi, who Mars rocks from ‘Earth rocks from 4 7 
directed the Curiosity mission. adryriverbed. a dry riverbed. 4 18. 
| L The surface of Mars may have 
looked different in ancient times 
than it does today. Describe one 
way. 

/ Student 

; 


But if Mars had an ancient sea, this raises another 
question, says Elachi: “Could life have evolved on Mars?” 
And if it could: “Did life evolve? And where is it now?” 


HEAT 
AND 
LIGHT 
YOUR 
HOME 
WITH 
THE SUNI 


Mars had rivers. 


The answer shown illustrates the type of student response that would receive full credit (1 point). 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Percent 


Countr 
rahe Full Credit 


Norway (5) 65 (1.7) © 
= Denmark 58 (1.6) © 
Sweden 57 (1.9) © 

3 Singapore 54 (13) O 

12 Canada 49 (1.5) © 

t United States 42 (1.5) 
Italy 42 (1.7) 
Slovenia 41 (1.6) 

3 Israel 37 (1.2) @ 
Chinese Taipei 34 (1.6) @ 
Ireland 32 (1.9) @ 

2 Portugal 31 (1.1) @ 

1 Georgia 26 (1.6) @ 
United Arab Emirates 24 (0.7) @ 

Benchmarking Participants 
Dubai, UAE 36 (1.1) @ 
Abu Dhabi, UAE 21 (1.1) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 
See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.5.2: Advanced International Benchmark for ePIRLS Online 
Informational Reading - Example Item 2 


Process: Make Straightforward Inferences 


PIRLS 


7 “The New Yorkntrmary a Class Project 


— ——— 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


Need for Doctors | Need for Patients 


Need for Patients j 
In 1851, Dr. Elizabeth Blackwell, the first Ae 14. 

woman to be allowed to practice medicine in os 7 : When Elizabeth Blackwell first 
America, rented a small room and opened a Seay opened her clinic in New York, 
clinic. She waited for patients, but no one : what shows that most people did 
came. Dr. Blackwell did not give up. She began not accept the idea of a woman 
offering free talks to women about how to doctor? 

keep themselves and their babies healthy. Student 

Slowly, people started coming to see her. Dr. Elizabeth Blackwell New York 


No one came to her clinic. 


The answer shown illustrates the type of student response that would receive full credit (1 point). 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Country Percent 
Full Credit 
3 Singapore 63 (1.3) O 
Ireland 48 (1.7) © 
Chinese Taipei 47 (1.6) © 
3 Israel 42 (13) © 
t United States 40 (1.6) O 
Norway (5) 39 (1.5) 
Italy 38 (1.4 
37 (04) 
12 Canada 35 (17 
= Denmark 32 (16) @ 
Sweden 32 (1.5) @ 
United Arab Emirates 25 (0.9) @ 
1 Georgia 24 (1.7) @ 
2 Portugal 24 (1.1) @ 
Slovenia 23 (14) @ 
Benchmarking Participants 
Dubai, UAE 37 (1.4 
Abu Dhabi, UAE 17 (1.0) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 


See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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PANTS 


Exhibit 2.5.3: Advanced International Benchmark for ePIRLS Online 
Informational Reading - Example Item 3 


Process: Interpret and Integrate Ideas and Information 


Integrate informat rom across multiple webpages to provide 3 objects matched to their functions 


ome 
PIRLS 


Class Project 


tl 
Home Getting to Mars Missions Seeking Signs of Life Rover Called Curiosity 
; 14. 
New inventions have helped 


scientists have sent three types of missions to Mars. scientists look at Mars from 
- locations closer and closer to the 


| Soa 

be s eee, ‘ wn planet. 

besa mee 4 eT upest Mare: hey cook as many es a9 In the boxes below each location, 
: es as possible as they went by. R write the name of the invention 

f that scientists used to look at 


Mars. You can look back at the 
webpages. 


Orbiters Student 
By the year 2000, countries were able to put spacecraft into orbit ; 


around Mars. Long-term studies were now possible. Today, 
several spacecrafts are still orbiting Mars. q Looking at Mars from Earth 


SOURCE: IEA's Progress in International Reading Literacy Study - PIRLS 2016 


nw 
n 
pee 
a 
QO 
te) 
4 
a 


Telescopes 


Rovers Looking at Mars from space 
In recent years, scientists thought of ways to put rovers on Mars. Flyby missions 

Arover is a remote-controlled vehicle with six wheels. It is the size a 
of a small car. It can travel around and explore the surface of é ooking at Mars from its surface 


Rovers 


The answer shown illustrates the type of student response that would receive full credit (3 points). 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Country Percent 
Full Credit 
lreland 37 (2.1) © 
Sweden 33 (1.9) © 
3 Singapore 31(14) © 
Norway (5) 30 (1.2) © 
t United States 26 (1.4) © 
12 Canada 26 (1.4) © 
Slovenia 25 (1.5) 
= Denmark 24 (1.8) 
2 Portugal 23 (1.3) 
Italy 16 (1.2) @ 
3 Israel 15 (1.2) @ 
Chinese Taipei 14 (1.0) @ 
United Arab Emirates 12 (0.6) @ 
1 Georgia 8 (1.2) @ 


Benchmarking Participants 
Dubai, UAE 23 (0.7) 
Abu Dhabi, UAE 7 (0.9) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 
See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.5.4: Advanced International Benchmark for ePIRLS Online 
Informational Reading - Example Item 4 


Process: Interpret and Integrate Ideas and Information 


Integrate information from a webpage to provide an explanation 


ePIRLS Online Reading 2016 
http://www.mars-exploration-program.org/missions PIRLS 


[ine Soin system / ars Exporaton Prooram 7 Class Project 


‘missions: imply flew past Mars. They took as many 
as possible as they went by. 


SOURCE: IEA's Progress in International Reading Literacy Study - PIRLS 2016 


By the year 2000, countries were able to put spacecraft into orbit j 
around Mars. Long-term studies were now possible. Today, ‘ 13. 


several spacecrafts are still orbiting Mars. Now that there are rovers on 


Mars, why are orbiters still useful? 


Student 
Rovers 
In recent years, scientists thought of ways to put rovers on Mars. E 


Arover is a remote-controlled vehicle with six wheels. It is the size f 
of a small car. It can travel around and explore the surface of ES GE ES OS ene 


Mars , _ changes over time on Mars. 


{ 


The answer shown illustrates the type of student response that would receive full credit (1 point). 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Percent 
Counciy Full Credit 
Norway (5) 42 (1.7) @ 
3 Singapore 38 (13) © 
t United States 35 (1.7) © 
Sweden 33 (16) O 
Slovenia 33 (13) O 
3 Israel 30 (14) © 
Ireland 27 (1.6 
7) 
12 Canada 26 (1.6 
Italy 22 (1.2) @ 
= Denmark 19 (15) @ 
United Arab Emirates 19 (0.9) @ 
1 Georgia 18 (1.7) @ 
2 Portugal 16 (0.9) @ 
Chinese Taipei 13 (0.9) @ 
Benchmarking Participants 
Dubai, UAE 28 (1.3 
Abu Dhabi, UAE 14 (14) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 
See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.5.5: Advanced International Benchmark for ePIRLS Online 
Informational Reading - Example Item 5 


Process: Evaluate and ique Content and Textual Elements 


The Gift of Curiosity 


By Maria Green 


For decades, scientists have wondered whether the 
surface of Mars ever had water. Now, Curiosity’s 
cameras show signs that Mars once had rivers. There 
are photographs of what looks like a dried riverbed. 
Other photographs show huge canyons and valleys that 
could have been made by rivers. 


SOURCE: IEA's Progress in International Reading Literacy Study - PIRLS 2016 


Scientists are now “convinced that Mars did have an 
ocean a few billion years ago,” says Charles Elachi, who Mars rocks from. Earth rocks from 
directed the Curiosity mission. adryriverbed. —_a dry riverbed. 


20. 

The writer of “The Gift of Curiosity” 
is in favor of exploring Mars. 
Explain how the writer shows this. 


Student 


But if Mars had an ancient sea, this raises another 
question, says Elachi: “Could life have evolved on Mars?” 
And if it could: “Did life evolve? And where is it now?” 


HEAT 
AND 
LIGHT 
YOUR 
HOME 
WITH 
THE SUNI 


She calls it "the gift." 


The answer shown illustrates the type of student response that would receive full credit (1 point). 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Percent 


Countr 
a Full Credit 


Ireland 48 (1.8) O 

Norway (5) 47 (1.4) © 

t United States 45 (1.8) © 

12 Canada 42 (1.6) © 

Slovenia 27 (14) O 
25.03) 

3 Israel 21 (13) @ 

Sweden 21 (13) @ 

3 Singapore 19 (0.9) @ 

= Denmark 19 (15) @ 

2 Portugal 18 (1.3) @ 

United Arab Emirates 13 (0.6) @ 

Italy 10 (0.9) @ 

1 Georgia 8 (0.7) @ 

Chinese Taipei 7 (08) @ 

Benchmarking Participants 

Dubai, UAE 25 (1.0 

Abu Dhabi, UAE 9 (1.1) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 


See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 2.5.6: Advanced International Benchmark for ePIRLS Online 
Informational Reading —- Example Item 6 


Process: Evaluate and Critique Content and Textual Elements 
Description: Evaluate the use of a timeline to convey informa 


tion 
: ———oaoooaom 79 > 


http:/www.womenshistory.org/dreblackwell : PIRLS 


[women's History Class Project 
| Timeline 1821-1910 


1821 ©@ Born3 February in Bristol, England, Elizabeth is the third 
of nine children. 


1832 ©@ Elizabeth's family moves to New York, America. 


1 84 7 @ Elizabeth is accepted into Geneva Medical College, New 


York to study medicine. 
| &. 
1849 Elizabeth finishes university at the top of her class and is What is an advantage of 
the first woman ever to receive a medical degree in presenting the events in 
. . Elizabeth's life in a timeline? 
America. Elizabeth travels to France where she works at 


La Maternité Hospital. Student 


Elizabeth travels back to America and opens up her own 
clinic, The New York Infirmary. 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


It gives detailed information 


Elizabeth trains nurses who will treat injured soldiers It shows the events in order of 
fighting in the Civil War in the United States. importance. 


Fi Bocas It lets readers choose what to 
Elizabeth returns to England and opens a clinic in saad fiat 
London, becoming the first female doctor in England. YI I evvvvvvvvvvvvsvovsssnssssnsnsnnnnnnnnnnnnnnnnnnnnnnnnnnsnnnenes 
It lists the events in the order 
they took place. 


Elizabeth helps to create the London School of Medicine 
for Women. 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Percent 
SUNNY, Correct 
Sweden 63 (1.8) © 
Slovenia 62 (1.5) © 
Italy 58 (15) O 
3 Israel 54 (1.4) © 
= Denmark 52 (1.7) © 
Ireland 52 (1.8) O 
3 Singapore 49 (1.4) 
Chinese Taipei 47 (1.4) 
12 Canada 47 (1.9) 
t United States 46 (1.7) 
Norway (5) 43 (1.7) @ 
2 Portugal 43 (1.6) @ 
United Arab Emirates 29 (0.8) @ 
1 Georgia 17 (1.2) @ 
Benchmarking Participants 
Dubai, UAE 37 (13) @ 
Abu Dhabi, UAE 23 (1.2) @ 


© Percent significantly higher than international average 
@ Percent significantly lower than international average 
See Appendix B.1 for target population coverage notes 1, 2, and 3. See Appendix B.4 for sampling guidelines and sampling participation notes t, +, and =. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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GHAPTER 3: 
Nee 355 \N/e) 
EXPERIENGE WITH 


DIGITAL DEVICES 


ePIRLS 2016 INTERNATIONAL RESULTS IN 
ONLINE INFORMATIONAL READING 


Z, TIMSS & FIRLS 
nternational Study Center 

Z IEA. Lynch S i =f Edvicatio 
BOSTON COLLECE 


Students in ePIRLS Familiar with Computers 
and the Internet 


Had Access to Digital Percent of 
Devices in the Home students [ 


Students with access to 
digital devices at home 
had higher ePIRLS 
achievement. 


High 


Average Achievement 
C4 
® 


Medium 
Low 
Attended Schools Percent of _ 
Unaffected by Digital students 
Resource Shortages 
Students in ePIRLS Not 


attended schools Affected 


well-resourced with 
digital devices. 


550 


Somewhat 
Affected | 500 


Average Achievement 


Affected 
aLot | 450 


Reported Self-Efficacy Percent of 
in Computer Use students 
Students with 

self-efficacy had 

higher ePIRLS High 


achievement. 


Medium 


Average Achievement 


Low 


Classroom Instruction Involving Computers to 
Prepare Reports 


Using a Computer to Prepare Reports for Schoolwork 


34% 
minutes minutes 
302: day 3 or less per day 


FOR AN AVERAGE ACHIEVEMENT SCORE OF HAD AN AVERAGE ACHIEVEMENT SCORE OF 


549 535 


Using a Computer to Find and Read Information on the Internet 


39% 
minutes minutes 
302: day 3 or less per day 


536 543 


Using computers in school to prepare reports was associated with higher 

b ePIRLS achievement, whereas finding and reading information on the 
Internet each day was not. It is not just the amount of time students use 
computers that makes a difference, but how they use them. 


SOURCE: IEA's Progress in International Reading yy TSS eS 
Literacy Study ~ PIRLS 2016 F IEA Lynch School of Education 
http://pirls2016.0rg/download-center/ BOSTON COLLEGE 


PIRLS 


CHAPTER 3 


Access and Experience with 
Digital Devices 


Exhibit 3.1 and 3.2: Home Resources for Learning 


The Home Resources for Learning scale combines data reported by students and their parents. The 
parents’ data were collected using the PIRLS 2016 Learning to Read Survey in which students’ 
parents were asked to provide information about their child’s experiences learning to read. As 
explained in Exhibit 3.1, students provided information about the number of books in the home 
and other study supports, while the parents provided information about the number of children’s 
books, the parents’ levels of education, and their occupations. As also explained, students were 
assigned a score on the scale according to the availability of these five home resources for learning. 

In Exhibit 3.1, ePIRLS countries are ordered by the percentage of students in the Many 
Resources category. However, on average, almost three-fourths of the students (71%) were assigned 
to the Some Resources category. Twenty-six percent were in the Many Resources category and 
only 3 percent in the Few Resources category. Students in the Many Resources category had higher 
achievement on ePIRLS than the students in the Some Resources category (577 vs. 530). 

Exhibit 3.2 presents information about students’ access to digital devices in the home. 
The percentages of students with High, Medium, and Low Access and their associated average 
achievement mirror the percentages with Many, Some, and Few Resources. 
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PXI(4 Capsrade 


Exhibit 3.1: Home Resources for Learning 


Students Categorized by Parents' and Students’ Reports 

Note: Results based on students who participated in both PIRLS and ePIRLS. 
Students were scored according to their own and their parents’ responses concerning the availability of five resources on the Home Resources 
for Learning scale. Students with Many Resources had a score of at least 11.8, which is the point on the scale corresponding to students 
reporting they had more than 100 books in the home and two home study supports, and parents reporting that they had more than 25 
children's books in the home, that at least one parent had finished university, and that at least one parent had a professional occupation, on 
average. Students with Few Resources had a score no higher than 7.5, which is the scale point corresponding to students reporting that they 


had 25 or fewer books in the home and neither of the two home study supports, and parents reporting that they had 10 or fewer children's 
books in the home, that neither parent had gone beyond upper-secondary education, and that neither parent was a small business owner or 


had a clerical or professional occupation, on average. All other students were assigned to the Some Resources category. 


Country Percent Average ePIRLS Percent Average ePIRLS Percent Average ePIRLS 


Average 
Scale Score 


of Students Achievement of Students Achievement of Students Achievement 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


Norway (5 46 (1.3) 588 (2.3) 54 (1.3 554 (2.3) (0.1) 

Sweden 43 (1.5) 589 (2.4) 56 (1.5) 546 (2.6) 0 (0.1) ~~ 

Denmark 40 (1.7) 586 (2.5) 59 (17 545 (2.3) (0.2) ~~ 

Canada r 34 (1.5) 578 (3.6) 65 (1.5) 533 (2.7) 0 (0.2) ~~ .06) 

Ireland 33 (1.5) 604 (2.4) 66 (1.6 557 (2.4) (0.3) ~~ .06) 

Singapore 29 (0.9) 634 (3.5) 69 (0.8) 574 (3.2) 2 (0.2) ~~ .03) 

Israel 22 (1.3) 586 (2.9) 76 (1.3 529 (2.6) 2 (0.2) ~~ .05) 

Slovenia 2 (1.1) 563 (2.8) 77 (1.1) 518 (2.1) 1 (0.2) ~~ .6 (0.04) 

Chinese Taipei 21 (1.3) 578 (2.4) 74 (1.2 540 (1.9) 5 (0.4) 497 (6.3) 10.3 (0.06) 

Portugal 18 (1.0) 561 (3.7) 76 (1.0) 518 (2.1) 6 (0.5) 483 (5.3) 10.1 (0.05) 

United Arab Emirates 13 (0. , 547 (4.0) 85 (0.5 468 (2.2) 3 (0.3) 408 (9.9) 10.2 (0.03) 

Georgia 12 (1.1 510 (5.1) 82 (1.4) 476 (3.2) 6 (0.9) 439 (7.9) 10.0 (0.06) 

Italy 8 (0: ' 575 (3.7) 86 (0.8 535 (1.8) 6 (0.6) 496 (6.4) 9.7 (0.05) 

United States -- -- -- 
Pintemationaiavs. | 250) | soy | rey sso | sf 4653) 
Benchmarking Participants 

Dubai, UAE 21 (0.5) 585 (2.2) 77 (0.5) 522 (1.8) 2 (0.1) ~~ 10.7 (0.02) 

Abu Dhabi, UAE r 10 (0.7) 513 (8.7) 87 (0.8) 436 (3.8) 3 (0.4) 358 (14.1) 10.0 (0.04) 


This PIRLS questionnaire scale was established in 2011 based on the combined response distribution of all countries that participated in PIRLS 2011. To provide a point 
of reference for country comparisons, the scale centerpoint of 10 was located at the mean of the combined distribution. The units of the scale were chosen so that 2 
scale score points corresponded to the standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
A dash (-) indicates comparable data not available. A tilde (~) indicates insufficient data to report achievement. 
An “r” indicates data are available for at least 70% but less than 85% of the students. 


Number of books in the home (students): 
1) 0-10 

2) 11-25 

3) 26-100 

4) 101-200 

5) More than 200 


Number of children’s books in the home (parents): 
1) 0-10 

2) 11-25 

3) 26-50 

4) 51-100 

5) More than 100 


Number of home study supports (students): 
1) None 

2) Internet connection or own room 

3) Both 


Highest level of education of either parent (parents): 


1) Finished some primary or lower secondary 
or did not go to school 

2) Finished lower secondary 

3) Finished upper secondary 

4) Finished post-secondary education 

5) Finished university or higher 


Highest level of occupation of either parent (parents): 


1) Has never worked outside home for pay, general laborer, or semi-professional (skilled agricultural or 
fishery worker, craft or trade worker, plant or machine operator) 

2) Clerical (clerk or service or sales worker) 

3) Small business owner 

4) Professional (corporate manager or senior official, professional, or technician or associate professional) 


eS 
Some Few 
Resources Resources 
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Exhibit 3.2: Digital Devices in the Home PAUSE C&D srace 


Students Categorized by Parents' and Students' Reports 

Note: Results based on students who participated in both PIRLS and ePIRLS. 
Students were scored according to their own and their parents’ responses concerning the availability of four items on the Digital Devices in the 
Home scale. Students with High Access had a score of at least 12.1, which is the point on the scale corresponding to students reporting that 
they had a computer and Internet connection, and parents reporting they had seven or more digital information devices in the home as well 
as a digital device for reading for both themselves and their child. Students with Low Access had a score no higher than 6.0, which is the scale 
point corresponding to students reporting that they did not have a computer or Internet connection, and parents reporting that they had less 
than four digital information devices in the home and no digital devices for reading for either themselves or their child. All other students 
were assigned to the Medium Access category. 


Country Percent Average ePIRLS Percent Average ePIRLS Percent Average ePIRLS 
of Students Achievement of Students Achievement of Students Achievement 


Average 


Scale Score 


SOURCE: IEA's Progress in International Reading Literacy Study - PIRLS 2016 


Norway (5) 58 (1.1) 574 (2.4) 42 (1.1 561 (2.7 0 (0.0) 11.8 (0.04) 
Denmark 49 (1.2) 565 (2.8) 51 (1.2) 557 (2.4) 0 (0.0) me 11.5 (0.05) 
Sweden 43 (1.3) 572 (2.9) 57 (1.2 557 (2.5 0 (0.1) a 11.3 (0.05) 
Canada t 27 (1.2) 564 (3.5) 72 (1.2) 542 (3.3) 0 (0.1) a 10.6 (0.05) 
United Arab Emirates 26 (0.6) 492 (3.0) 73 (0.5 469 (2.5 1 (0.1) ~~ 10.5 (0.03) 
Ireland 25 (1.1) 584 (3.1) 75 (1.1) 566 (2.7) 0 (0.1) ao 10.4 (0.04) 
Singapore 24 (0.6) 619 (3.1) 76 (0.6 582 (3.2 1 (0.1) ne 10.4 (0.03) 
Israel 23 (0.9) 551 (3.5) 76 (0.9) 539 (2.7) 1 (0.3) we 10.3 (0.04) 
Portugal 21 (0.9) 545 (4.0) 78 (0.9 518 (2.1 1 (0.2) oe 10.4 (0.03) 
Italy 14 (0.7) 551 (3.9) 85 (0.8) 534 (2.0) 2 (0.3) we 9.8 (0.04) 
Slovenia 13 (0.7) 551 (3.6) 86 (0.6) 523 (2.0 1 (0.2) ~~ 9.9 (0.03) 
Chinese Taipei 11 (0.5) 569 (3.8) 87 (0.5) 544 (2.1) 2 (0.2) ad 9.7 (0.03) 
Georgia 4 (0.3) 505 (6.3) 87 (1.0) 479 (3.1 9 (1.0) 464 (9.4) 9.0 (0.06) 


United States -- -- Be od = te Bie 
International Avg. 26 (0.2) ES YAU) ve (0s) 536 (0.7) 1 (0.1) 464 (9.4) 


Benchmarking Participants 


Dubai, UAE 30 (0.5) 544 (2.6) 70 (0.5) 528 (1.8) 0 (0.1) ies 10.8 (0.02) 
Abu Dhabi, UAE iF 26 (1.0) 462 (5.6) 73 (1.0) 432 (4.2) 1 (0.1) ms 10.5 (0.04) 


This PIRLS questionnaire scale was established in 2016 based on the combined response distribution of all countries that participated in PIRLS 2016. To provide a point of 
reference for country comparisons, the scale centerpoint of 10 was located at the mean of the combined distribution. The units of the scale were chosen so that 2 scale score 
points corresponded to the standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
A dash (-) indicates comparable data not available. A tilde (~) indicates insufficient data to report achievement. 
An “r” indicates data are available for at least 70% but less than 85% of the students. 


Home computer/tablet or Internet connection Digital device for reading (parents): 
(students): 1) None 

1) None 2) Either for the parent or for the child 
2) Computer/tablet or Internet connection 


Number of digital information devices in 
the home (parents): 

1) None 

2) 1-3 devices 

3) 4-6 devices 


Medium Low 
Access Access 
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Exhibit 3.3: Instruction Affected by Digital Resources Shortages - 

Principals’ Reports 

Exhibit 3.3 presents the results for the Digital Resource Shortages scale, with the questions comprising 
the scale provided below the results. Countries are ordered according to the percentage of students 
(from most to least) in schools Not Affected by resource shortages, from a high of 57 to a low of 9 
percent. On average, 33 percent of the ePIRLS students attended well-resourced schools and they 
had the highest average achievement (548). Sixty percent of the students were in schools Somewhat 
Affected by resource shortages and 7 percent in schools Affected A Lot. For ePIRLS, average 
reading achievement for these two groups was similar, 536 and 539. 
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Exhibit 3.3: Instruction Affected by Digital Resource Shortages — 


Principals' Reports 


Students Categorized by Principals' Reports 
Note: Results based on students who participated in both PIRLS and ePIRLS. 


Students were scored according to their principals’ responses concerning four school and classroom resources on the Digital Resource 
Shortages scale. Students in schools where instruction was Not Affected by resource shortages had a score on the scale of at least 11.3, which 


corresponds to their principals reporting that shortages affected instruction “not at al 


two resources, on average. Students in schools where instruction was Affected A Lot had a score no higher than 7.2, which corresponds to 
their principals reporting that shortages affected instruction “a lot” for two of the four resources and “some” for the other two resources, on 
average. All other students attended schools where instruction was Somewhat Affected by resource shortages. 


Not Affected Somewhat Affected Affected A Lot 


Percent 
of Students 


Percent 
of Students 


Average ePIRLS 
Achievement 


\" 


for two of the four resources and “a little” for the other 


Average ePIRLS 
Achievement 


PIRLS 


Reading 


PANS 


Average 


Percent 
of Students 


Average ePIRLS 
Achievement 


Scale Score 


SOURCE: IEA's Progress in International Reading Literacy Study - PIRLS 2016 


Singapore 57 (0.0) 583 (4.2) 37 (0.0 592 (5.6 6 (0.0) 11.6) 11.2 (0.00 
Sweden 52 (4.3) 566 (3.6) 47 (4.3) 552 (2.7) 1 (0.8) ~~ 11.3 (0.16) 
Slovenia 49 (5.2) 527 (3.3) 50 (5.1 523 (3.0 1 (0.4) ~~ 11.1 (0.14) 
Canada 46 (4.8) 546 (4.4) 49 (4.9) 540 (4.1) 5 (1.4) 539 (7.1) 10.8 (0.18) 
United States 44 (4.7) 557 (4.3) 52 (4.9 555 (4.4) 3 (1.0) 591 (17.7) 10.9 (0.17, 
Denmark 43 (4.2) 561 (3.5) 56 (4.5) 555 (3.1) 1 (0.8) a 11.0 (0.13) 
Norway (5) 35 (4.2) 572 (3.7) 62 (4.3 565 (2.7 3 (1.3) 567 (14.3) 10.7 (0.12 
United Arab Emirates 30 (1.7) 508 (5.1) 55 (2.3) 449 (3.6) 15 (1.7) 464 (8.1) 9.8 (0.10) 
Ireland 26 (4.0) 575 (3.9) 69 (4.1 565 (3.0 5 (1.9) 543 (8.1) 10.2 (0.15 
Georgia 25 (3.1) 485 (6.5) 74 (3.1) 475 (4.2) 1 (0.7) ~~ 10.1 (0.11) 
Israel 17 (3.2) 555 (5.3) 69 (3.9 539 (3.9 14 (2.9) 499 (8.3) 9.3 (0.18 
Chinese Taipei 15 (3.2) 551 (4.7) 71 (4.3) 545 (2.4) 14 (3.1) 542 (5.7) 9.3 (0.14) 
Portugal 10 (2.0) 546 (9.2) 75 (3.3 521 (2.1 15 (3.1) 513 (6.3) 9.1 (0.14 
Italy 9 (2.3) 540 (6.0) 78 (3.5) 532 (2.7) 13 (3.0) 531 (5.7) 8.9 (0.13) 
Benchmarking Participants 
Dubai, UAE 45 (0.3) 544 (2.4) 38 (0.3) 508 (2.2) 18 (0.3) 531 (4.5) 10.3 (0.02) 
Abu Dhabi, UAE 27 (3.4) 479 (10.9) 62 (3.9) 416 (4.7) 11 (2.4) 405 (16.2) 9.8 (0.16) 


This PIRLS questionnaire scale was established in 2016 based on the combined response distribution of all countries that participated in PIRLS 2016. To provide a point of 
reference for country comparisons, the scale centerpoint of 10 was located at the mean of the combined distribution. The units of the scale were chosen so that 2 scale score 
points corresponded to the standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
A tilde (~) indicates insufficient data to report achievement. 


How much is your school’s capacity to provide instruction affected by a shortage or inadequacy of 
the following? 


Not at all Alittle Some 


A. General School Resources 

Technologically competent staff 

Audio-visual resources for delivery of instruction 
(e.g., interactive white boards, digital projectors) 
Computer technology for teaching and learning 
(e.g., computers or tablets for student use) 


B. Resources for Reading Instruction 


1) Computer software/applications for 


O 


reading instruction 


Somewhat 
Affected 


Not 
Affected 
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Exhibit 3.4: Students Use Computers or Tablets for Finding and 

Presenting Information 

Exhibit 3.4 presents the results for three questions students were asked about how they used 
computers and tablets more than 30 minutes a day. The first question pertained to using these devices 
to prepare reports and presentations. About one-third (34%) of the students reported that they 
spent more than 30 minutes a day using computers and tablets to prepare reports and presentations, 
and they had higher ePIRLS achievement than their counterparts (549 vs. 535). The next question 
was about using the devices for schoolwork to find information and read information, and only 
19 percent reported that they did so. These 19 percent had somewhat lower achievement than 
their counterparts (534 v. 541), which is consistent with some research showing schools encourage 
more frequent tablet use for lower achieving students. Finally, the third question asked simply 
about reading information on the Internet each day. Here, 39 percent of the students responded 
affirmatively, but they also had somewhat lower ePIRLS achievement (536 vs. 543). It could be the 
case that students are reading on the Internet about many different types of information. 


Exhibit 3.4: Students Use Computers or Tablets for Finding and 
Presenting Information 
Students’ Reports 
Note: Results based on students who participated in both PIRLS and ePIRLS. 
Using a Computer or Tablet for Schoolwork ona 
Normal School Day to Prepare Reports and 
Presentations 


Using a Computer or Tablet for Schoolwork on a 
Normal School Day to Find and Read Information 


Finding and Reading Information 
on the Internet Each Day 


Percent of | Average ePIRLS | Average ePIRLS} Percentof | Average ePIRLS | Average ePIRLS| Percentof | Average ePIRLS | Average ePIRLS 
Students Who | Achievement for} Achievement | Students Who} Achievement for} Achievement | Students Who| Achievement for] Achievement 
Spend More | StudentsWho | forStudents | SpendMore | StudentsWho | forStudents | SpendMore | StudentsWho | for Students 
Than 30 —_| Spend More Than} Who Spend 30 Than30 —_| Spend More Than| Who Spend 30 Than 30 —_| Spend More Than| Who Spend 30 


Minutes 30Minutes | MinutesorLess) Minutes 30Minutes | MinutesorLess] Minutes 30Minutes | Minutes or Less 


Country 


Canada 39 (1.6) 554 (3.1) 537 (4.6) 22 (0.8) 540 (3.2) 544 (3.9) 36 (1.4) 538 (2.7) 548 (4.2) 
Chinese Taipei 13 (0.6) 554 (3.9) 545 (2.1) 13 (0.7) 538 (3.9) 547 (2.1) 32 (1.0) 544 (2.7) 547 (2.1) 
Denmark 34 (1.5) 566 (3.3) 555 (2.5) 15 (1.1) 548 (4.3) 561 (2.4) 27 (1.0) 552 (3.5) 561 (2.4) 
Georgia 44 (1.5) 489 (3.1) 472 (3.6) 35 (1.4) 481 (3.7) 477 (3.5) 54 (1.2) 478 (3.3) 479 (4.2) 
Ireland 24 (1.4) 577 (3.6) 566 (2.7) 11 (0.8) 554 (6.0) 569 (2.4) 34 (0.9) 562 (3.1) 570 (3.0) 
Israel 55 (1.1) 551 (2.8) 521 (3.3) 30 (0.9) 534 (2.9) 538 (2.8) 48 (1.1) 531 (2.9) 544 (2.7) 
Italy 20 (1.2) 530 (3.2) 534 (2.2) 11 (0.8) 521 (4.8) 535 (2.1) 40 (1.1) 530 (2.9) 536 (2.1) 
Norway (5) 40 (2.0) 577 (2.8) 563 (2.5) 11 (0.7) 563 (4.2) 569 (2.2) 20 (0.8) 566 (3.4) 569 (2.2) 
Portugal 33 (1.1) 527 (2.2) 521 (2.6) 16 (0.8) 519 (4.1) 523 (2.5) 44 (0.9) 521 (2.3) 525 (2.7) 
Singapore 42 (0.8) 606 (2.9) 576 (3.4) 25 (0.6) 584 (3.7) 590 (3.0) 47 (0.8) 582 (3.2) 595 (3.2) 
Slovenia 36 (1.7) 527 (2.8) 525 (2.4) i (0.7) 516 (4.4) 527 (1.9) 38 (1.5) 520 (2.6) 529 (2.0) 
Sweden 21 (1.7) 567 (3.3) 560 (2.4) 16 (1.8) 555 (5.5) 561 (2.2) 24 (1.2) 558 (4.2) 562 (2.1) 
United Arab Emirates 38 (0.7) 492 (2.3) 460 (2.7) 25 (0.5) 479 (2.7) 468 (2.4) 60 (0.5) 468 (2.7) 479 (2.3) 
United States 35 (1.3) 570 (3.4) 554 (2.7) 21 (0.8) 542 (3.9) 561 (2.5) 38 (1.2) 550 (3.4) 563 (2.6) 


International Avg. 34 (0.4) By (I:)) 535 (0.8) 19 (0.3) 534 (1.1) 541 (0.7) BEA (\s)) 536 (0.8) 543 (0.8) 


Benchmarking Participants 


SOURCE: IEA's Progress in International Reading Literacy Study - PIRLS 2016 


Abu Dhabi, UAE 37 (1.3) 458 (4.8) 424 (4.6) 25 (1.0) 441 (5.7) 433 (4.1) 60 (1.0) 429 (4.4) 447 (4.3) 
Dubai, UAE 43 (0.7) 548 (2.0) 517 (1.9) 27 (0.6) 537 (2.4) 526 (1.6) 57 (0.8) 528 (1.8) 532 (2.1) 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 3.5: Self-Efficacy for Computer Use 

Exhibit 3.5 presents the results for the ePIRLS 2016 Self-Efficacy for Computer Use scale, based on 
how good students reported that they were at 1) using computers, 2) typing, and 3) looking up 
information on the Internet. Internationally, on average, 51 percent of the ePIRLS students reported 
High Self-Efficacy in using computers, 41 percent reported having Medium Self-Efficacy, and 
8 percent reported having Low Self-Efficacy. The High and Medium Self-Efficacy students had 
similar ePIRLS average achievement (546 vs. 541), but this was higher average achievement than 
the Low Self-Efficacy students (514). 
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Exhibit 3.5: Self-Efficacy for Computer Use PAUSE 
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pa 
AY) 
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0) 


Students' Reports 

Note: Results based on students who participated in both PIRLS and ePIRLS. 
Students were scored on the Self-Efficacy for Computer Use scale according to their responses to three statements about how good 
they were at using a computer. Students who had High Self-Efficacy for computer use had a score on the scale of at least 10.4, 
which corresponds to “agreeing a lot” with two of the three statements and “agreeing a little” with the other one, on average. 
Students who had Low Self-Efficacy for using a computer had a score no higher than 7.1, which corresponds to students 
“disagreeing a little” to two of the three statements and “agreeing a little” with the other one, on average. All other students had 
Medium Self-Efficacy for using a computer. 


High Self-Efficacy Medium Self-Efficacy Low Self-Efficacy 
Country Percentof | Average ePIR Percentof | Average ePIR Percent of | Average ePIRLS 


Average 


Scale Score 
Students Achievement Students Achievement Students Achievement 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


Israel 65 (1.2) 544 (2.6) 30 (1.0) 534 (3.5) 5 (0.5) 496 (8.7) 10.6 (0.05) 
Portugal 64 (1.1) 526 (2.3) 32 (1.0) 521 (2.5) 4 (0.4) 486 (8.7) 10.6 (0.04) 
Slovenia 61 (1.2) 529 (2.1) 34 (1.1) 526 (2.6) 5 (0.4) 509 (7.8) 10.4 (0.05) 
Denmark 55 (1.4) 567 (2.6) 39 (1.3) 554 (2.7) 5 (0.4) 525 (6.0) 10.2 (0.05) 
Ireland 55 (1.3) 571 (2.8) 39 (1.2) 571 (3.1) 6 (0.6) 542 (7.4) 10.1 (0.06) 
Norway (5) 54 (1.2) 573 (2.4) 42 (1.2) 568 (2.6) 4 (0.4) 534 (7.6) 10.2 (0.04) 
Italy 52 (1.1) 534 (2.4) 40 (1.0) 536 (2.7) 8 (0.7) 522 (4.8) 10.1 (0.05) 
United Arab Emirates 52 (0.6) 490 (2.7) 41 (0.5) 471 (2.4) 7 (0.3) 425 (4.5) 10.0 (0.03) 
United States 51 (1.2) 563 (2.8) 42 (1.1) 560 (3.3) 7 (0.5) 536 (5.1) 10.0 (0.05) 
Sweden 49 (1.3) 565 (2.7) 45 (1.1) 565 (2.4) 6 (0.6) 529 (5.5) 10.0 (0.05) 
Georgia 45 (1.4) 493 (3.6) 45 (1.1) 479 (3.5) 10 (0.8) 462 (7.1) 9.7 (0.06) 
Singapore 40 (0.7) 595 (2.9) 49 (0.6) 590 (3.3) 11 (0.5) 567 (5.0) 9.5 (0.03) 
Canada 39 (0.8) 550 (3.5) 52 (0.9) 547 (3.7) 9 (0.9) 523 (8.4) 9.5 (0.04) 
Chinese Taipei 35 (0.9) 550 (2.6) 46 (0.9) 548 (2.5) 19 (0.8) 536 (3.1) 9.1 (0.04) 
Benchmarking Participants 
Dubai, UAE 55 (0.8) 541 (1.5) 40 (0.7) 528 (2.3) 5 (0.3) 488 (5.5) 10.2 (0.03) 
Abu Dhabi, UAE 51 (1.1) 458 (5.0) 41 (1.0) 437 (4.5) 9 (0.6) 388 (7.7) 9.9 (0.05) 


This ePIRLS questionnaire scale was established in 2016 based on the combined response distribution of all countries that participated in ePIRLS 2016. To provide a 
point of reference for country comparisons, the scale centerpoint of 10 was located at the mean of the combined distribution. The units of the scale were chosen so 
that 2 scale score points corresponded to the standard deviation of the distribution. 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


How much do you agree with the following statements? 


Agree a lot Agree a little Disagree Disagree 
| a little alot 
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1) lam good at using a computer 


2) lam good at typing 


3) Itis easy for me to find information on 
the Internet 


Self- 


Medium t Low Self-Efficacy 
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Students in ePIRLS Successful in Webpage Navigation 


Followed Percent of 
Teacher Avatar’s students 
Instruction 


Most students navigated 
to all required webpages. 


Students Were Not perish of 
Distracted by students 
Advertisements 


Most students did not 
click on the advertisements 
embedded in the 
ePIRLS tasks. 
No Clicks 


One or 
More Clicks 


Students Percent of 
Completed students 
All Questions 


In ePIRLS, most students 
had no difficulty 
completing the 
assessment. 


Percent of 


Looking Back students 


Across Webpages 
to Integrate 
Information 


Students who looked 

back had higher 
achievement, but most 
students did not look back. 
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CHAPTER 4 


Navigation Through ePIRLS 


Exhibit 4.1: Navigation to Required Webpages 

To keep students on track, the teacher avatar gave periodic instructions about what webpage students 
should find. The results in Exhibit 4.1 indicate that students had little difficulty navigating using 
the ePIRLS tabs and links. Ninety-three percent followed the teacher avatar’s instruction, clicking 
on these webpages on the first click. These students had substantially higher achievement than 
the students who could not navigate by themselves to the webpages (544 vs. 476). As a point of 
information, students who became lost were automatically supplied with the correct webpages. 


Exhibit 4.1: Navigation to Required Webpages 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Navigated to All Did Not Navigate to All 
Required Webpages Required Webpages 


Count 
meets Average parent Average 
ePIRLS ePIRLS 
of Students ; of Students ; 
Achievement Achievement 


Percent 


= 

R 

3 
Singapore 99 (0.1) 589 (3.0) 1 (0.1) we 2 
Ireland 97 (0.4) 569 (2.5) 3 (0.4) 497 (7.1) 3 
Sweden 97 (0.3) 561 (2.2) 3 (0.3) 493 (6.4) 
Norway (5) 97 (0.3) 570 (2.2) 3 (0.3) 508 (5.5) & 
United States 97 (0.4) 559 (2.5) 3 (0.4) 478 (5.4) . 
Chinese Taipei 96 (0.3) 547 (2.0) 4 (0.3) 516 (5.6) = 
Canada 96 (0.3) 546 (3.0) 4 (0.3) 473 (5.6) ¥ 
Denmark 95 (0.3) 561 (2.3) 5 (0.3) 502 (5.3) > 
Slovenia 94 (0.4) 528 (1.9) 6 (0.4) 467 (5.3) a 
Portugal 93 (0.4) 526 (2.2) 7 (0.4) 478 (3.2) ri 
Italy 92 (0.5) 537 (2.1) 8 (0.5) 486 (3.5) ij 
Israel 90 (0.5) 545 (2.2) 10 (0.5) 457 (4.6) 2 
United Arab Emirates 84 (0.6) 483 (2.2) 16 (0.6) 397 (3.4) ° 
Georgia 76 (1.1) 489 (3.2) 24 (1.1) 439 (3.9) 

Benchmarking Participants 

Dubai, UAE 92 (0.3) 536 (1.5) 8 (0.3) 429 (4.1) 
Abu Dhabi, UAE 80 (1.1) 448 (4.0) 20 (1.1) 369 (5.0) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
A tilde (~) indicates insufficient data to report achievement. 
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Exhibit 4.2: Navigation to Advertisements 

Students working on a school assignment or research project will finish sooner if they focus on 
finding critical information and are not distracted. To collect information about this, a number 
of the webpages in ePIRLS contained advertisements. Almost identical to the results in Exhibit 
4.1, 92 percent of the students remained focused and did not click on any advertisements (average 
achievement of 542). Eight percent did click on at least one advertisement and had lower average 
achievement (512). 


Exhibit 4.2: Navigation to Advertisements 
Note: Results based on students who participated in both PIRLS and ePIRLS. 


One or More Clicks 


No Clicks on Advertisements 


on Advertisements 


Country 


Percent 
of Students 


Average 
ePIRLS 
Achievement 


Percent 
of Students 


Average 
ePIRLS 


Achievement 


2 
R 
z 
e 
me} 
3 
Sweden 96 (0.3) 560 (2.2) 4 (0.3) 542 (6.9) 2 
Slovenia 95 (0.3) 526 (2.0) 5 (0.3) 509 (4.8) 3 
Canada 95 (0.5) 545 (3.0 5 (0.5) 507 (8.8) = 
Denmark 94 (0.4) 559 (2.2) 6 (0.4) 541 (5.0) § 
Ireland 94 (0.6) 568 (2.3 6 (0.6) 550 (9.5) . 
Portugal 94 (0.4) 525 (2.2) 6 (0.4) 485 (5.1) = 
Italy 93 (0.4) 534 (2.1 7 (0.4) 509 (4.7) ¥ 
Norway (5) 93 (0.4) 568 (2.1) 7 (0.4) 557 (5.0) > 
United States 92 (0.6) 558 (2.4) 8 (0.6) 540 (6.2) a 
Singapore 91 (0.3) 588 (3.0) 9 (0.3) 588 (5.0) rs 
Israel 90 (0.5) 543 (2.2 10 (0.5) 477 (6.1) 8 
Georgia 90 (0.5) 481 (3.1) 10 (0.5) 438 (6.0) 3 
United Arab Emirates 86 (0.4) 481 (2.2) 14 (0.4) 391 (3.1) 7 
Chinese Taipei 82 (0.7) 548 (2.0) 18 (0.7) 535 (3.3) 
Benchmarking Participants 
Dubai, UAE 91 (0.3) 533 (1.4) 9 (0.3) 472 (4.7) 
Abu Dhabi, UAE 82 (0.7) 448 (4.2) 18 (0.7) 357 (5.0) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Exhibit 4.3: Students Reaching All Items in ePIRLS Tasks 

The results in Exhibit 4.3 show that ePIRLS provided about the right amount of time for most 
students. On average, 91 percent of the students reached all of the ePIRLS items. As might be 
anticipated, the 91 percent completing the assessment had higher average achievement than the 9 
percent who did not (543 and 497, respectively). 


Exhibit 4.3: Students Reaching All Items in ePIRLS Tasks 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Reached All Items Did Not Reach All Items 


Average Average 
Country Percent aa Percent ie 
of Students of Students 


Achievement Achievement 


© 

R 

2 
Singapore 98 (0.1) 589 (3.0) 2 (0.1) ~~ s 
Norway (5) 96 (0.4) 570 (2.2) 4 (0.4) 519 (4.2) a 
Sweden 95 (0.4) 562 (2.3) 5 (0.4) 516 (4.6) 5 
Ireland 95 (0.6) 569 (2.5) 5 (0.6) 513 (6.8) & 
United States 95 (0.4) 559 (2.5) 5 (0.4) 503 (5.3) & 
Chinese Taipei 95 (0.4) 547 (2.0) 5 (0.4) 524 (4.7) FE: 
Slovenia 93 (0.5) 528 (1.9) 7 (0.5) 489 (4.8) 2 
Denmark 92 (0.5) 562 (2.2) 8 (0.5) 513 (3.9) s 
Canada 92 (0.6) 546 (3.1) 8 (0.6) 503 (5.5) 
Italy 88 (0.7) 536 (2.3) 12 (0.7) 510 (3.0) & 
Portugal 87 (0.6) 525 (2.3) 13 (0.6) 503 (3.0) = 
Israel 86 (0.6) 545 (2.3) 14 (0.6) 483 (3.4) i 
United Arab Emirates 83 (0.7) 476 (2.4) 17 (0.7) 435 (2.8) 2 
Georgia 71 (1.4) 485 (3.4) 29 (1.4) 457 (3.8) g 

Benchmarking Participants 

Dubai, UAE 91 (0.3) 534 (1.6) 9 (0.3) 467 (3.0) 
Abu Dhabi, UAE 83 (1.2) 436 (4.5) 17 (1.2) 411 (4.0) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
A tilde (~) indicates insufficient data to report achievement. 
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Exhibit 4.4: Students Look Back at Prior Webpages when Answering 

Integrate Items 

To collect information about how well students could integrate information across websites, there 
were six items that specifically required students to provide information from prior websites together 
with information from the current website. Exhibit 4.4 provides the averages in “looking back” 
navigation across the six items, and then three examples from the released tasks are provided in 
Exhibits 4.4.1 through 4.4.3. 

Exhibit 4.4 contains a considerable amount of information, so it is accompanied by a diagram. 
The first set of columns in Exhibit 4.4 show that on average, only 17 percent of the students looked 
back when answering the integrate questions and their average achievement was 558. The “look 
back” students were among the better achieving ePIRLS students (referencing top-performing 
Singapore with average achievement of 588). Interestingly, of the 17 percent that looked back, 7 
percent answered correctly on average and 10 percent did not. The achievement for these groups 
was 589 (similar to Singapore) for the students that answered correctly and 540 for the students who 
answered incorrectly despite looking back. 

The second set of columns show that, on average, the 83 percent of the students that did not 
look back had average achievement of 535. This is lower than the students who did look back 
by 23 points. However, it is not low performance, so it is likely that some students either knew 
or could remember information across websites. Still, only 28 percent of the “did not look back” 
students answered these questions correctly, on average, and their average achievement was 572. It 
is clear, that looking back could have helped some students. In total, 55 percent of the students, on 
average, did not look back to prior webpages and did not answer correctly. These students had lower 
achievement than their counterparts (517). 

In summary, the students that looked back and answered correctly, either because they were 
double checking or because they successfully found the answer, had the highest achievement (589) 
followed by the students who knew the answer without looking back (572). Next were the students 
who tried to look back but were not successful in answering correctly (540). Unfortunately, the 
majority of students did not look back when it may have helped. On average, the students who did 
not look back and answered incorrectly had the lowest average achievement on ePIRLS (517). 

Example Item 4.4.1 is from the “Mars” task. Students needed to remember or look back to an 
earlier webpage to find that scientists were looking at Mars through telescopes, even in the days 
before space exploration. Perhaps most students thought they remembered, because hardly any of 
them “looked back”—only 6 percent. Ninety-four percent did not look back, but only 22 percent 
actually answered correctly. 

On Example Item 4.4.2 from “Elizabeth Blackwell”, one-third of the students “looked back” and 
it helped 8 percent answer correctly. This 8 percent had very high ePIRLS achievement (610). Of the 
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67 percent that did not look back, only 10 percent answered correctly. So once again, the majority 
(58%) did not look back and did not answer correctly. 

Finally, results were similar on another item about Elizabeth Blackwell’s accomplishments. 
Twenty-seven percent “looked back” and 7 percent answered correctly compared to 20 percent 
who did not (average achievement 597 and 545, respectively). Of the 73 percent who did not look 
back, 12 percent answered correctly (average achievement 575) and the remaining 60 percent did 
not (average achievement 522). 


Exhibit 4.4: Students Look Back at Prior Webpages when Answering Integrate Items 


Students Who Did Not Look Back 
Answered Look Back Answered Look Back 
Average Items Correctly Items Incorrectly 
Percent of 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Students Who Looked Back 
Answered Look Back Answered Look Back 
ener t Average Items Correctly Items Incorrectly 


Country 
ePIRLS Average Average 
Achievement ePIRLS ePIRLS 
Achievement Achievement 


ePIRLS Average Average 


Achievement ePIRLS ePIRLS 
i Students i 
Achievement Achievement 


Students Students 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


Canada 14 (0.6) 563 (4.9) 0.3) 4.9) 0.4) 5.6) 86 (0.6) 540 (3.1) 31 (0.6) 575 (2.9) 0.8) 2.9) 
Chinese Taipei 37 (0.7) 560 (2.0) 16 (0.4) 581 (1.9) 21 (0.5) 544 (2.3) 63 (0.7) 536 (2.2) 17 (0.4) 568 (1.9) 46 (0.7) 523 (2.3) 
Denmark 14 (0.5) 571 (3.3) 5 (0.4) 597 (3.8) 9 (0.4) 557 (4.0) 86 (0.5) 557 (2.3) 31 (0.8) 589 (2.1) 55 (0.8) 539 (2.4) 
Georgia 9 (0.4) 500 (5.3) 30.2) 547 (5.9) 6 (0.3) 481 (5.6) 91 (0.4) 475 (3.3) 18 (0.7) 522 (2.9) 73 (0.8) 464 (3.3) 
Ireland 17 (0.6) 586 (3.7) 8 (0.4) 613 (3.3) 9 (0.4) 562 (4.5) 83 (0.6) 563 (2.7) 35 (0.7) 593 (2.4) 48 (0.8) 541 (2.8) 
Israel 18 (0.5) 558 (2.9) 8 (0.4) 592 (3.4) 10 (0.4) 533 (3.4) 82 (0.5) 531 (2.4) 28 (0.6) 579 (2.3) 54 (0.8) 507 (2.5) 
Italy 16 (0.7) 548 (2.6) 6 (0.3) 573 (3.0) 10 (0.4) 535 (2.9) 84 (0.7) 530 (2.2) 26 (0.5) 559 (1.9) 58 (0.8) 517 (2.3) 
Norway (5) 16 (0.6) 586 (2.7) 6 (0.4) 607 (3.4) 9 (0.4) 571 (2.8) 84 (0.6) 564 (2.4) 34 (0.7) 591 (2.2) 51 (0.9) 546 (2.5) 
Portugal 20 (0.5) 534 (2.9) 6 (0.2) 559 (2.7) 14 (0.4) 524 (3.4) 80 (0.5) 519 (2.3) 22 (0.5) 552 (2.6) 58 (0.6) 507 (2.1) 
Singapore 23 (0.5) 619 (2.8) 13 (0.5) 641 (2.7) 10 (0.3) 591 (3.2) 77 (0.5) 579 (3.2) 34 (0.5) 612 (2.6) 43 (0.9) 552 (3.5) 
Slovenia 17 (0.7) 540 (2.3) 5 (0.2) 567 (4.1) 12 (0.6) 530 (2.6) 83 (0.7) 521 (2.0) 23 (0.5) 557 (2.0) 60 (0.8) 507 (2.1) 
Sweden 13 (0.6) 579 (2.5) 5 (0.3) 600 (3.1) 8 (0.4) 567 (3.0) 87 (0.6) 557 (2.4) 35 (0.7) 585 (2.1) 52 (1.0) 537 (2.4) 
United Arab Emirates 13 (0.3) 497 (3.3) 4(0.1) 569 (2.4) 9 (0.2) 467 (3.7) 87 (0.3) 464 (2.2) 18 (0.4) 538 (1.8) 69 (0.5) 445 (2.2) 
United States 14 (0.5) 577 (3.6) 6 (0.4) 604 (4.0) 8 (0.3) 555 (4.0) 86 (0.5) 553 (2.6) 38 (0.7) 586 (2.1) 48 (0.9) 528 (2.6) 
Benchmarking Participants 
Abu Dhabi, UAE 13 (0.4) 453 (6.1) 30.2) 550 (6.0) 10 (0.3) 426 (5.9) 87 (0.4) 429 (4.0) 14 (0.5) 513 (4.3) 74 (0.7) 414 (3.7) 
Dubai, UAE 17 (0.4) 559 (2.2) 70.2) 595 (2.5) 10 (0.2) 533 (2.7) 83 (0.4) 522 (1.6) 27 (0.5) 568 (1.3) 56 (0.4) 499 (1.9) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 


Average Percent of Students Who Answered Look Back Items Correctly and Incorrectly 


LOOK 
BACK 


CORRECT INCORRECT CORRECT INCORRECT 
Ey:}") 540 572 517 
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Exhibit 4.4.1: Students Look Back at Prior Webpages when Answering 
Integrate Items - Example Item 1 
Note: Results based on students who participated in both PIRLS and ePIRLS. 


Students Who Looked Back 


Answered Look Back Answered Look B 
Average Item Correctly Item Incorrectly 
Percent of 


Students Who Did Not Look Back 


Answered Look Back Answered Look Back 
Average Item Correctly Item Incorrectly 


ePIRLS Average Average 
: Percent of 
Achievement ePIRLS Galiky 
Students ; ; 
Achievement Achievement 


Country 
ePIRLS Average Average 
Achievement ePIRLS ePIRLS 
Achievement Achievement 


Students 


SOURCE: IEA's Progress in International Reading Literacy Study - PIRLS 2016 


Canada 30.4) 554 (13.8) 1(0.2) 611 (13.3) 2 (0.3) 539 (16.1) 97 (0.4) 542 (3.2 25 (1.4) 598 (3.5 71 (1.4) 523 (3.5) 
Chinese Taipei 14 (1.2) 555 (4.3) 2 (0.4) 605 (5.6) 11 (1.0) 545 (4.4) 86 f 2) 540 (2.5 11 (0.9) 599 (3.6 75 (1.4) 531 (2.4) 
Denmark 9 (1.3) 559 (9.9) 2 (0.4) 618 (9.1) 7 (1.3) 544 (9.8) 91 (1.3) 556 (3.0 22 (1.7) 612 (3.5 69 (1.7) 538 (2.8) 
Georgia 2 (0.3) 462 (15.3) 0 (0.2) 522 (16.9) 2 (0.3) 449 (17.5) 98 (0.3) 476 (3.8) 8 (1.2) 559 (7.1) 90 (1.2) 469 (3.5) 
Ireland 4 (0.8) 570 (8.5) 1 (0.3) 612 (9.9) 3 (0.7) 557 (10.7) 96 (0.8) 569 (3.3) 36 (2.1) 613 (3.5 60 (2.0) 541 (3.0) 
Israel 2 (0.4) 557 (19.8) 0 (0.2) 621 (26.6) 1 (0.3) 534 (23.7) 98 (0.4) 534 (2.8) 15 (1.2) 600 (4.1) 83 (1.2) 523 (3.0) 
Italy 4 (0.6) 555 (7.9) 0 (0.2) 604 (19.3) 3 (0.6) 548 (8.1) 96 (0.6) 532 (2.5 16 (1.1) 582 (3.6) 80 (1.2) 522 (2.6) 
Norway (5) 6 (0.8) 587 (6.2) 1 (0.3) 612 (9.1) 4 (0.7) 579 (7.2) 94 (0.8) 570 (2.7) 28 (1.3) 617 (3.3) 66 (1.4) 550 (2.8) 
Portugal 7 (0.6) 525 (4.9) 2 (0.3) 564 (9.5) 6 (0.6) 513 (5.8) 93 (0.6) 522 (2.6 22 (1.3) 571 (3.3 71 (1.2) 508 (2.5) 
Singapore 7 (0.6) 608 (5.6) 2 (0.4) 649 (6.4) 4 (0.5) 585 (6.0) 93 (0.6) 585 (3.3 28 (1.3) 642 (2.8) 65 (1.4) 561 (3.5) 
Slovenia 5 (0.7) 557 (7.7) 2 (0.4) 593 (9.7) 4 (0.6) 539 (8.2) 95 (0.7) 526 (2.7 24 (1.4) 582 (2.9) 71 (1.6) 507 (2.8) 
Sweden 6 (1.0) 578 (9.3) 2 (0.5) 629 (9.5) 4(0.7) 546 (9.6) 94 (1.0) 560 (3.0 31 (1.9) 607 (2.8 63 (1.8) 536 (3.2) 
United Arab Emirates : 0.5) 503 (7.9) 1(0.2) 605 (7.1) 5 (0.4) 482 (8.7) 94 (0.5) 467 (2.5 11 (0.6) 591 (3.1 83 (0.6) 451 (2.7) 
United States a ee 572) ue 3) a ul 647 (15.5) 2 (0. 547 (15. 97 (0. d 556 (2. 7 26 7 7 613 mr 7 72 (1 7 535 (3. 7 
Benchmarking Participants 
Abu Dhabi, UAE 6 (0.7) 463 (18.2) 1(0.2) 588 (15.7) 5 (0.7) 440 (18.8) 94 (0.7) 428 (4.7) 6 (0.8) 580 (8.4) 89 (1.0) 419 (4.3) 
Dubai, UAE 9 (0.7) 556 (6.1) 2 (0.3) 610 (8.1) 7 (0.6) 539 (7.0) 91 (0.7) 528 (2.0) 20 (0.8) 607 (2.5) 71 (0.8) 505 (2.7) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Seeking Signs of Life 
14. 


a New inventions have helped 

t have sent three types of missions to Mars. scientists look at Mars from 
locations closer and closer to the 
planet. 
In the boxes below each location, 
write the name of the invention 
that scientists used to look at 
Mars. You can look back at the 
webpages. 


Orbiters: WE j j Student 
ei 2000, countries were able to put spacecraft into orbit i Z 


flev v past Mars. They took as many 


4 ite ind Mars. Long-term studies were now possible. Today, 


iS 


| spacecrafts are still orbiting Mars. Looking at Mars from Earth 


Telescopes 


Rovers Looking at Mars from space 
In recent years, scientists thought of ways to put rovers on Mars. 4 Flyby missions 

Arover isa remote-controlled vehicle with six wheels. It is the size 
of a small car. It can travel around and explore the surface of 
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Exhibit 4.4.2: Students Look Back at Prior Webpages when Answering 
Integrate Items - Example Item 2 
Note: Results based on students who participated in both PIRLS and ePIRLS. 


Students Who Looked Back 


Answered Look Back Answered Look Back 
Derentnt Average Item Correctly Item Incorrectly 
ePIRLS Average Average 
Percent of| 


Achievement ePIRLS ePIRLS 
Students 


Students Who Did Not Look Back 
Answered Look Back Answered Look Back 
Average Item Correctly Item Incorrectly 


Country 
ePIRLS Average Average 
Achievement ePIRLS ePIRLS 
Achievement Achievement 


Students 


Achievement 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


Canada ; 572 (5.0) 8 (1. A) 559 (5.9 69 (1.9) 531 (3.4 11 (1.1) 601 (5.4) 59 (1. 3.3) 
Chinese Taipei 50 (1.6) 569 (2.4 7 (0.6) 618 (5.0) 42 (1.5) 560 (2.8) 50 (1.6) 518 (3.2) 2 (0.3) 592 (13.1) 49 (1.6) 515 (3.2) 
Denmark 31(1.4) 578 (4.5 7 (0.8) 612 (6.7) 24 (1.3) 569 (5.2 69 (1.4) 551 (3.4 12 (1.2) 607 (6.1) 57 (1.6) 539 (3.7) 
Georgia 18 (1.2) 508 (5.6) 4 (0.6) 563 (10.3) 14(1.1) 492 (6.3) 82 (1.2) 474 (4.4) 13 (1.3) 537 (5.3) 68 (1.8) 461 (4.4) 
Ireland 28 (1.9) 600 (4.5 9 (1.0) 638 (6.1) 19 (1.4) 584 (6.0 72 (1.9) 557 (4.0 13 (1.1) 615 (4.5) 59 (1.9) 544 (4.3) 
Israel 37 (1.1) 569 (3.9 15 (1.0) 609 (4.2) 22 (1.1) 542 (4.4) 63 (1.1) 515 (3.2) 14 (1.1) 593 (5.0) 49 (1.2) 492 (3.5) 
Italy 27°(1.5) 554 (3.5 5 (0.7) 593 (6.9) 22 (1.2) 545 (3.5 73 (1.5) 526 (2.7 11 (0.9) 569 (5.2) 63 (1.5) 519 (2.7) 
Norway (5) 26 (1.4) 592 (4.3) 6 (0.8) 631 (7.8) 20 (1.2) 581 (4.1) 74 (1.4) 557 (3.4) 9 (0.7) 611 (4.5) 64 (1.4) 550 (3.6) 
Portugal 44 (1.2) 539 (3.3 5 (0.6) 584 (6.1) 38 (1.3) 533 (3.8 56 (1.2) 506 (2.7 7 (0.8) 560 (6.8) 49 (1.4) 499 (2.7) 
Singapore 54 (1.5) 621 (2.6 30 (1.2) 645 (2.6) 24 (1.0) 592 (3.3) 46 (1.5) 550 (4.1) 13 (0.6) 601 (4.4) 33 (1.6) 531 (4.4) 
Slovenia 36 (1.8) 545 (3.3 3(0.4) 596 (9.1) 33 (1.7) 541 (3.5 64 (1.8) 509 (2.3 4(0.7) 558 (10.5) 60 (1.6) 506 (2.3) 
Sweden 23 (1.4) 585 (3.9 2 (0.5) 625 (9.1) 21 (1.3) 581 (4.1) 77 (1.4) 551 (3.0) 4(0.7) 623 (7.2) 72 (1.6) 547 (2.9) 
United Arab Emirates 22 (0.6) 511 (4.0 5 (0.4) 597 (4.4) 17 (0.6) 485 (4.7 78 (0.6) 459 (2.7 12 (0.6) 541 (3.3) 66 (0.8) 444 (2.6) 
United States 32 (1.5) 581 (4.4 10 (1.1) 617 (6.9) 22 (1.1) 564 (4.9) 68 (1.5) 542 (3.1) 13 (0.9) 600 (5.1) 55 (1.7) 529 (2.9) 
Benchmarking Participants 
Abu Dhabi, UAE 21 (1.1) 472 (6.9 4 (0.5) 586 (8.1) 17 (1.0) 444 (7.3) 79 (1.1) 424 (4.9) 9 (0.9) 527 (6.9) 70 (1.4) 411 (4.6) 
Dubai, UAE 30 (0.7) 566 (3.5 10 (0.8) 609 (4.2) 20 (0.8) 544 (5.0) 70 (0.7) 514 (2.2) 17 (1.2) 570 (3.9) 53 (1.0) 496 (2.0) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Elizabeth's Goal Applying | Studying | Practicing 


A LADY DOCTOR? 
You Must Be Joking! 


10. 
Pra cticing : Think of everything you have read 
Within a few years of Elizabeth leaving college, more than 20 women throughout the H J on the website “A Lady Doctor? 


country were studying in medical schools, including Elizabeth's younger sister, Emily. i ~ You must be joking!” 

H Give two examples that show how 
Unlimited free Elizabeth Blackwell did not give up 

calls for the first and kept trying. 


America, however, was still not ready for a “lady doctor.” No hospital in America 
would hire Elizabeth, so she travelled to France to work in a hospital there. Elizabeth 
worked hard and started studying to become a surgeon. However, she became ill 


cep 3 2 : month with Student 
and lost sight in one eye and had to give up her dream of becoming a surgeon. : Roverphone 
Despite this, she kept working hard as a general doctor. H 


She kept applying to 
medical schools even 
after they rejected her 


She kept studying even 
when she wasn't allowed 


‘ : =" in practice classes 
Photo Gallery ie 
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Students Look Back at Prior Webpages when Answering 
Integrate Items - Example Item 3 
Note: Results based on students who participated in both PIRLS and ePIRLS. 


Canada 22 (1.6) 566 (5.9) 4 (0.5) 605 (7.7) 18 (1.4) 557 (6.3) 78 (1.6) 537 (3.4) 12 (1.2) 581 (5.1) 66 (1.9) : 529 (3.4) 


Chinese Taipei 57 (1.4) 555 (2.6) 13 (0.9) 582 (4.4) 44 (1.3) 547 (3.0) 43 (1.4) 527 (3.4) 5 (0.6) 551 (5.4) 38 (1.4) 523 (3.6) 
Denmark 17 (1.4) 567 (6.2) 30.5) 606 (12.1) 14 (1.3) 558 (6.5) 83 (1.4) 558 (3.3) 17 (1.3) 593 (5.4) 65 (1.7) 548 (3.5) 
Georgia 12 (1.1) 497 (11.8) 2 (0.6) 570 (16.0)  10(0.8) 482 (10.7) 88 (1.1) 478 (3.9) 9 (1.1) 536 (6.2) 78 (1.1) 470 (4.2) 
Ireland 32 (1.8) 588 (7.1) 13 (1.3) 623 (6.0) 19 (1.6) 564 (9.1) 68 (1.8) 560 (3.8) 19 (1.6) 601 (4.9) 49 (2.0) 545 (4.7) 
Israel 27 (1.6) 561 (4.2) 8 (0.8) 595 (6.2) 18 (1.2) 545 (5.6) 73 (1.6) 526 (3.1) 13 (1.1) 591 (5.0) 60 (1.7) 512 (3.1) 
Italy 26 (1.5) 544 (4.1) 6 (0.7) 575 (8.2) 21 (1.2) 535 (4.4) 74 (1.5) 529 (2.5) 12 (0.9) 564 (5.1) 62 (1.6) 523 (2.6) 
Norway (5) 26 (1.6) 580 (4.7) 8 (1.0) 613 (6.9) 18 (1.3) 565 (4.7) 74 (1.6) 562 (3.2) 18 (1.1) 591 (4.1) 56 (1.7) 552 (3.5) 
Portugal 31 (1.7) — 536 (4.8) 4 (0.6) 574 (7.3) 27 (1.6) 530 (5.4) 69 (1.7) 514 (2.5) 5 (0.6) 565 (7.2) 64 (1.8) 510 (2.5) 
Singapore 39 (1.3) 623 (3.2) 17 (0.9) 645 (4.6) 23 (1.0) 607 (3.7) 61 (1.3) 566 (3.6) 20 (0.9) 600 (3.4) 41 (1.2) 550 (4.2) 
Slovenia 28 (1.4) 537 (3.8) 5 (0.6) 567 (9.3) 22 (1.4) 530 (4.4, 72 (1.4) 516 (2.5) 6 (0.7) 563 (6.5) 66 (1.5) 512 (2.5) 
Sweden 22 (1.6) 579 (4.2) 3 (0.6) 613 (9.1) 18 (1.6) 573 (4.4 78 (1.6) 554 (3.0) 9 (0.8) 594 (4.6) 69 (1.7) 549 (3.2) 
United Arab Emirates 20 (0.7) 500 (4.7) 4 (0.3) 577 (5.4) 16 (0.7) 478 (5.4 80 (0.7) 463 (2.5) 10 (0.5) 536 (4.6) 70 (0.8) 453 (2.6) 


United States 22 (1.2) 577 (5.4) 7 (0.7) 615 (6.4) 15 (1.0) 561 (6.1 78 (1.2) 548 (3.2) 19 (1.1) 586 (4.9) 59 (1.4) 536 (3.4) 
Benchmarking Participants 


Abu Dhabi, UAE 19 (1.3) 448 (9.1) 30.5) 545 (14.0) 16 (1.1) 430 (10.1) 81 (1.3) 431 (4.7) 7 (0.8) 495 (10.0) 74 (1.5) 424 (4.7) 
Dubai, UAE 26 (1.0) 567 (2.7) 8 (0.6) 604 (4.4) 17 (0.8) 548 (3.6 74 (1.0) 517 (2.3) 16 (0.9) 571 (3.4) 59 (1.1) 502 (2.7) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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17. 

Think about all of the information 
-# you have read about Elizabeth 

Need for Doctors | Need for Patients A New Hospital | Blackwell. Give three of her 


important achievements. 


=e — @™® ¢ Student 
A New Hospital i P 
Within seven years, Dr. Blackwell had raised enough money - as” / ve 


to open a small hospital called The New York Infirmary. Her —— 
sister Emily, also a doctor and a surgeon, performed om ; ii She got into medical 
operations. “ ao > hakactt 


Dr. Blackwell's goals were to: 


1. take care of the poor, particularly women and children She became the first woman 
2. train nurses ee doctor. 
3. provide a place for women doctors to work. 


The name and address of the hospital Dr. Blackwell started 
has changed many times since her days but it is still an 
important hospital in New York. The hospital still trains 
doctors and nurses, and takes care of the poor in the area. 
Also, the hospital is still dedicated to advancing women’s 
health. Today, leading doctors from all over the world 
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Appendix A.1: Distribution of Assessment Items by Comprehension Process and 
Item Format 


Multiple-Choice Items Constructed Response Items Total Items Percentage of 
ePIRLS Assessment Items 


Score Points 


| Comprehension Process Process 


Focus on and Retrieve 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 


10 10 12 13 22 23 21% 

Explicitly Stated Information ° 
Make Straightforward 1 n 15 19 7 31 28% 
Inferences 
Interpret and Integrate Ideas 

‘ 7 1 16 27 23 38 34% 
and Information 
Evaluate and Critique Content 1 "1 8 9 19 0 18% 
and Textual Elements 
Items 40 44 51 68 91 112 100% 
Percentage of Score Points 39% 61% 


Because of rounding some results may appear inconsistent. 
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Appendix B.1: Coverage of ePIRLS 2016 Target Population 


Ko) 

International Target Population Exclusions from National Target Population R 

Countr he = 
Exclusions Exclusions = 

Students from the provinces of British Columbia, a 

Re eAnaee ie Newfoundland, Ontario, and Quebec as 30% oon iS 
Chinese Taipei 100% 0.0% 0.9% 0.9% =I 
Denmark 100% 1.9% 8.0% 9.9% 5 

1 Georgia 96% Students taught in Georgian and Azerbaijani 0.8% 3.0% 3.8% 4 
Ireland 100% 2.3% 14% 3.7% E 

3 Israel 100% 21.0% 3.9% 24.9% 3 
Italy 100% 0.8% 4.1% 4.9% 2 
Norway (5) 100% 2.0% 3.4% 5.3% £ 

2 Portugal 100% 1.0% 6.5% 75% e 
3 Singapore 100% 10.6% 0.5% 11.1% & 
Slovenia 100% 1.5% 0.8% 2.4% 2 
Sweden 100% 1.3% 3.9% 5.2% , 
United Arab Emirates 100% 2.0% 1.3% 3.3% & 
United States 100% 0.0% 4.9% 4.9% 8 


Benchmarking Participants 


Abu Dhabi, UAE 100% 2.2% 1.7% 3.9% 
Dubai, UAE 100% 1.6% 1.6% 3.2% 


1 National Target Population does not include all of the International Target Population. 
2 National Defined Population covers 90% to 95% of National Target Population. 


3 National Defined Population covers less than 90% of the National Target population (but at least 77%). 
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Appendix B.2: School Sample Sizes 


: ae Number of Schools in 5 

Number of Schools in Number of Eligible Schools ee Number of Replacement Total Number of Schools [is 

Country - . Original Sample that ae .h a 
Original Sample in Sample ae Schools that Participated that Participated = 

Participated a 

Canada 507 503 467 7 474 2 
Chinese Taipei 150 150 150 0 150 o 
Denmark 198 191 132 10 142 g 
Georgia 201 201 197 2 199 2 
Ireland 150 148 147 0 147 3 
Israel 160 160 155 2 157 - 
Italy 150 150 133 15 148 s 
Norway (5) 153 152 138 4 142 E 
Portugal 222 221 211 7 218 = 
Singapore 177 177 177 0 177 rn 
Slovenia 122 170 159 0 159 5 
Sweden 158 154 144 0 144 7 
United Arab Emirates 482 475 464 1 465 s 
United States 176 1722 128 25 153 i 
ac 

Benchmarking Participants 3 


Abu Dhabi, UAE 153 151 150 0 150 
Dubai, UAE 178 175 174 0 174 
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Appendix B.3: Student Sample Sizes 


oO 

Within-school Number of Winter g 

Student Sampled Number of Numberof [i 

ek : Students Number of Number of = 

Country Participation Students in ; Students a Students f 

. eee Withdrawn from Eligible Students | Students Absent = 

(Weighted Participating Excluded Assessed s 

Class/School a 

Percentage) Schools = 

Canada 10,178 9,704 8,871 2 

Chinese Taipei 98% 4471 39 38 4,394 95 4,299 o 

Denmark 87% 3,139 48 219 2,872 366 2,506 8 

Georgia 95% 6,072 58 128 5,886 329 5,557 = 

Ireland 91% 2,767 18 44 2,705 232 2,473 s 

Israel 91% 4315 14 105 4,196 398 3,798 € 

Italy 92% 4,295 22 166 4,107 340 3,767 = 

Norway (5) 88% 4,294 48 136 4,110 500 3,610 - 

Portugal 92% 5,305 58 293 4,954 396 4,558 5 

Singapore 95% 6,719 29 0 6,690 370 6,320 ot 

Slovenia 93% 4,676 10 35 4,631 328 4303 a 

Sweden 90% 4528 34 170 4,324 445 3,879 a 

United Arab Emirates 92% 17,208 89 232 16,887 1,321 15,566 2 

United States 90% 4,884 155 175 4,554 464 4,090 A 

Benchmarking Participants 

Abu Dhabi, UAE 92% 4,367 20 27 4,320 340 3,980 
Dubai, UAE 92% 8,302 50 148 8,104 633 7471 


Students attending a sampled class at the time the sample was chosen but leaving the class before the assessment was administered were classified as “withdrawn.” 
Students with a disability or language barrier that prevented them from participating in the assessment were classified as “excluded.” 
Students not present when the assessment was administered, and not subsequently assessed in a make-up session, were classified as “absent.” 


In schools with 21 or fewer 4th grade students, all PIRLS students were selected to participate in ePIRLS; in larger schools, a subset of PIRLS students was randomly 
selected. 
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Appendix B.4: Participation Rates (Weighted) 


School Participation Overall Participation 
Eaante Class Student 
un 
. Before After Participation | Participation After 
Replacement Replacement Replacement 


Canada 79% 85% 100% 93% 74% 79% 
Chinese Taipei 100% 100% 100% 98% 98% 98% 
= Denmark 67% 74% 100% 87% 58% 64% 
Georgia 97% 99% 100% 95% 92% 94% 
Ireland 99% 99% 100% 91% 91% 91% 
Israel 97% 98% 100% 91% 88% 89% 
Italy 89% 99% 100% 92% 82% 91% 
Norway (5) 91% 93% 99% 88% 79% 81% 
Portugal 97% 99% 100% 92% 90% 91% 
Singapore 100% 100% 100% 95% 95% 95% 
Slovenia 94% 94% 99% 93% 86% 86% 
Sweden 93% 93% 99% 90% 83% 83% 
United Arab Emirates 98% 98% 100% 92% 90% 90% 
t United States 74% 89% 100% 90% 67% 80% 


SOURCE: IEA's Progress in International Reading Literacy Study — PIRLS 2016 8 
G) 
pm | 
fe) 
Qa 
fo) 


Benchmarking Participants 


Abu Dhabi, UAE 99% 99% 100% 92% 91% 91% 
Dubai, UAE 99% 99% 99% 92% 91% 91% 


PIRLS guidelines for sampling participation: The minimum acceptable participation rates were 85 percent of both schools and students, or a combined 
rate (the product of school and student participation) of 75 percent. Participants not meeting these guidelines were annotated as follows: 

t+ Met guidelines for sample participation rates only after replacement schools were included. 

+ Nearly satisfied guidelines for sample participation rates after replacement schools were included. 


= Did not satisfy guidelines for sample participation rates. 
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Appendix C.1: Average Percent Correct in the Comprehension Processes 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


Overall ePIRLS Comprehension Processes 


Online — - 
Retrieving and | Interpreting, 
Straightforward} Integrating, and 
Inferencing Evaluating 


Countr 
y Informational 


Reading 


< 

FS 

2 
Canada 59 (0.8) 64 (0.8) 5109) § 
Chinese Taipei 60 (0.5) 66 (0.5) 50 (0.5) 3 
Denmark 62 (0.6) 69 (0.5) 53 (0.7) 3 
Georgia 39 (0.8) 49 (0.9) 30(08) = 
Ireland 65 (0.7) 70 (0.6) 58 (0.7) § 
Israel 56 (0.6) 63 (0.6) 48 (0.6) E 
Italy 55 (0.6) 63 (0.6) 46 (06) = 
Norway (5) 65 (0.6) 71 (0.5) 58 (0.7) 
Portugal 52 (0.6) 60 (0.6) 43(06) & 
Singapore 70 (0.7) 76 (0.7) 62 (08) & 
Slovenia 53 (0.5) 60 (0.5) 44 (0.5) a 
Sweden 63 (0.6) 69 (0.6) 54 (07) 
United Arab Emirates 41 (0.5) 48 (0.5) 33 (0.5) 2 
United States 62 (0.7) 67 (0.6) 56(07) 

Benchmarking Participants 

Abu Dhabi, UAE 34 (0.8) 40 (0.9) 26 (0.8) 
Dubai, UAE 55 (0.3) 61 (0.4) 47 (03) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Appendix D.1: Percentiles of Online Informational Reading Achievement 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


5th 10th 25th 50th 75th 90th 95th 


Countr 
2, Percentile Percentile Percentile Percentile Percentile Percentile Percentile 


Canada 411 (5.8) 444 (5.3) 497 (4.6) 549 (3.6) 595 (3.0) 632 (3.5) 655 (4.9) 8 
Chinese Taipei 424 (5.4) 457 (3.9) 508 (2.7) 552 (2.0) 591 (2.7) 624 (2.6) 642 (3.1) 2 
Denmark 445 (6.8) 472 (3.8) 515 (3.0) 563 (3.2) 604 (3.4) 640 (3.2) 659 (3.8) 5 
Georgia 349 (6.5) 380 (6.4) 430 (4.2) 482 (4.2) 530 (3.8) 567 (4.0) 5873.1) oF 
Ireland 441 (7.7) 477 (5.6) 525 (3.4) 573 (2.6) 616 (2.9) 650 (2.7) 670 (4.9) 5 
Israel 386 (7.9) 424 (5.8) 485 (4.2) 545 (2.4) 595 (2.2) 634 (3.7) 655 (3.8) « 
Italy 425 (4.7) 452 (3.9) 493 (3.0) 536 (2.4) 576 (2.0) 609 (2.9) 628 (3.5) & 
Norway (5) 459 (5.4) 485 (3.9) 528 (3.6) 571 (2.5) 610 (2.6) 645 (3.1) 665 (3.9) 8 
Portugal 414 (4.7) 438 (3.4) 480 (2.8) 525 (2.4) 567 (2.0) 602 (2.7) 622 (3.8) 2 
Singapore 447 (7.5) 486 (6.0) 541 (3.7) 596 (3.3) 643 (3.3) 681 (2.9) 702 (4.2) = 
Slovenia 402 (5.3) 433 (3.6) 482 (2.7) 531 (2.4) 573 (2.4) 606 (2.8) 627 (2.3) é 
Sweden 440 (5.9) 473 (3.7) 520 (3.3) 565 (2.5) 604 (2.5) 637 (2.9) 657 (2.4) 3 
United Arab Emirates 291 (5.0) 327 (4.0) 398 (3.2) 476 (3.1) 542 (2.7) 595 (2.5) 625 (2.4) 2 
United States 426 (5.4) 458 (4.3) 510 (3.7) 562 (3.1) 608 (3.1) 647 (2.6) 669 (3.7) - 


Benchmarking Participants 
Gn 
Abu Dhabi, UAE 261 (5.2) 294 (6.1) 356 (6.2) 434 (5.0) 506 (4.7) 568 (6.2) 600 (5.5) 


Dubai, UAE 360 (4.4) 407 (3.6) 475 (3.1) 538 (1.6) 590 (2.0) 633 (1.8) 657 (2.3) 


Note: Percentiles are defined in terms of percentages of students at or below a point on the scale. 
() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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Appendix D.2: Standard Deviations of Online Informational Reading Achievement 


Note: Results based on students who participated in both PIRLS and ePIRLS. 


2 

a 

country Standard Standard Standard > 
Deviation Deviation Deviation & 

Canada : 1. : 74 (2.2) 539 (3.7 75 (1.8) 8 
Chinese Taipei 546 (2.0) 66 (1.3) 551 (2.3) 63 (1.7) 541 (2.2) 68 (1.5) re) 
Denmark 558 (2.2 66 (1.3) 560 (2.9 66 (2.1) 556 (2.9 66 (1.9) 3 
Georgia 477 (3.3) 73 (1.6) 485 (3.2) 70 (1.7) 469 (3.8) 75 (19) = 
Ireland 567 (2.5 71 (1.8) 572 (2.8 68 (2.0) 561 (3.4 72 (2.7) 8 
Israel 536 (2.3) 82 (1.9) 542 (2.5) 77 (2.2) 530 (3.1) 86 (2.2) € 
Italy 532 (2.1 62 (1.2) 534 (2.6 59 (1.4) 531 (2.4 64 (15) = 
Norway (5) 568 (2.2) 63 (1.2) 576 (2.6) 60 (1.7) 558 (2.9) 65 (1.7) . 
Portugal 522 (2.2 63 (1.2) 524 (2.6 63 (1.2) 521 (2.6 64 (1.6) 2 
Singapore 588 (3.0) 78 (2.1) 599 (3.2) 74 (2.1) 578 (3.3) 80 (25) & 
Slovenia 525 (1.9 68 (1.2) 532 (2.5 65 (2.0) 518 (2.5 70 (13) & 
Sweden 559 (2.3) 65 (1.1) 567 (2.6) 64 (1.3) 552 (2.7) 65 (1.7) Hi 
United Arab Emirates 468 (2.2 101 (1.4) 483 (3.4 92 (2.0) 454 (4.1 108 (1.9) 2 
United States 557 (2.6) 74 (1.3) 560 (2.8) 72 (1.5) 554 (3.1) it) he 


Benchmarking Participants 


Abu Dhabi, UAE 431 (4.1) 103 (2.5) 451 (6.6) 97 (3.0) 414 (6.5) 106 (2.9) 
Dubai, UAE 528 (1.6) 89 (1.2) 534 (2.7) 84 (1.9) 522 (2.8) 93 (2.3) 


() Standard errors appear in parentheses. Because of rounding some results may appear inconsistent. 
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APPENDIX E 


Organizations and Individuals 
Responsible for ePIRLS 2016 


Introduction 


PIRLS (Progress in International Reading Literacy Study) is a collaborative effort involving hundreds 
of individuals around the world. This appendix acknowledges the individuals and organizations 
who contributed to the assessment. Given that work on PIRLS 2016 has spanned approximately 
five years and has involved so many people and organizations, this list may not include all who 
contributed. Any omission is inadvertent. PIRLS 2016 also acknowledges the students, parents, 
teachers, and school principals who contributed their time and effort to the study. It would not be 
possible without them. 


Management and Coordination 


PIRLS is a major undertaking of IEA, and together with TIMSS (Trends in International Math and 
Science Study), comprises the core of IEA’ regular cycles of studies. The PIRLS assessment at the 
fourth grade complements TIMSS, which regularly assesses science and math achievement at the 
fourth and eighth grades. 

PIRLS was conducted by IEA’s TIMSS & PIRLS International Study Center at Boston College, 
which has responsibility for the overall direction and management of the TIMSS and PIRLS projects, 
including design, development, and implementation. Headed by Executive Directors Drs. Ina VS. 
Mullis and Michael O. Martin, the study center is located in the Lynch School of Education. In 
carrying out the project, the TIMSS & PIRLS International Study Center worked closely with IEA 
Amsterdam, which managed country participation, was responsible for verification of all translations 
produced by the participating countries, and coordinated the school visits by International Quality 
Control Monitors. Staff at IEA Hamburg worked closely with participating countries to organize 
sampling and data collection operations and to check all data for accuracy and consistency within 
and across countries; Statistics Canada in Ottawa was responsible for school and student sampling 
activities; The Australian Council for Educational Research (ACER) participated in developing 
the ePIRLS tasks and items, and ACER and the National Foundation for Educational Research in 
England (NFER) participated in developing the PIRLS 2016 passages and items; and Educational 
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Testing Service in Princeton, New Jersey consulted on psychometric methodology, provided software 
for scaling the achievement data, and replicated the achievement scaling for quality assurance. 

The Project Management Team, comprising the study directors and representatives from the 
TIMSS & PIRLS International Study Center, IEA Amsterdam and IEA Hamburg, Statistics Canada, 
and ETS met twice a year throughout the study to discuss the study’s progress, procedures, and 
schedule. In addition, the study directors met with members of IEA’s Technical Executive Group 
twice yearly to review technical issues. 

To work with the international team and coordinate within-country activities, each participating 
country designates an individual to be the PIRLS National Research Coordinator (NRC). The NRCs 
have the challenging task of implementing PIRLS in their countries in accordance with the PIRLS 
guidelines and procedures. In addition, the NRCs provide feedback and contributions throughout 
the development of the PIRLS assessment. The quality of the PIRLS assessment and data depends 
on the work of the NRCs and their colleagues in carrying out the complex sampling, data collection, 
and scoring tasks involved. Continuing the tradition of exemplary work established in previous 
cycles of PIRLS, the PIRLS 2016 NRCs performed their many tasks with dedication, competence, 
energy, and goodwill. 


Funding 


Funding for PIRLS 2016 was provided primarily by the participating countries. The National 
Center for Education Statistics of the U.S. Department of Education was a major funding partner, 
providing funding under contract number ED08C00117. The content of this publication does not 
necessarily reflect the views or policies of the U.S. Department of Education nor does mention of 
trade names, commercial products, or organizations imply endorsement by the U.S. Government. 
Boston College also is gratefully acknowledged for its generous financial support and stimulating 
educational environment. 
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TIMSS & PIRLS International Study Center at Boston College 

Ina V.S. Mullis, Executive Director 

Michael O. Martin, Executive Director 

Pierre Foy, Director of Sampling, Psychometrics, and Data Analysis 

Paul Connolly, Director, Graphic Design and Publications 

Marcie Bligh, Manager, Events and Administration 

Katie Trong Drucker, PIRLS Coordinator (through 2013) 

Susan Farrell, Lead Web and Database Designer 

Bethany Fishbein, Research Specialist, Instrument Development and Reporting 

Elena Forzani, Assistant Research Director, PIRLS (from 2015) 

Joseph Galia, Lead Statistician/Programmer 

Shirley Goh, Assistant Director, Communications and Media Relations 

Christine Hoage, Manager of Finance 

Kathleen Holland, Administrative Coordinator 

Martin Hooper, Assistant Research Director, TIMSS and PIRLS Questionnaire Development 
and Policy Studies 

leva Johansone, Associate Research Director, Operations and Quality Control 

Cristian Leiva, Front-End Web Developer 

Lauren Palazzo, Research Associate, TIMSS and PIRLS Questionnaire and Technical Reporting 

Yenileis Pardini, Lead Designer/Developer for eAssessments 

Mario Pita, Lead Graphic Designer 

Jyothsna Pothana, Statistician/Programmer 

Betty Poulos, Administrative Coordinator (through 2016) 

Caroline Prendergast, Research Associate, PIRLS 

Ruthanne Ryan, Senior Graphic Designer 

Jennifer Moher Sepulveda, Data Graphics Specialist (through 2015) 

Steven A. Simpson, Senior Graphic Designer 

Erin Wry, Research Associate, TIMSS and PIRLS Operations and Quality Control 

Liqun Yin, Research Psychometrician 


IEA Amsterdam 

Dirk Hastedt, Executive Director 

Hans Wagemaker, Executive Director (through 2014) 

Paulina Korsnakova, Director of the IEA Secretariat (through 2016) 
Andrea Netten, Director of the IEA Secretariat 

Barbara Malak, Manager, Member Relations (through 2013) 

Roel Burgers, Financial Director 
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Juriaan Hartenberg, Financial Manager (through 2013) 
Gabriela Nausica Noveanu, Senior Research Advisor 
David Ebbs, Senior Research Officer 

Michelle Djeki¢, Research and Liaison Officer 

Isabelle Braun-Gémin, Financial Officer 

Dana Vizkova, Financial Officer 

Gillian Wilson, Publications Officer 

Manuel Butty, Public Outreach Officer (through 2016) 


IEA Hamburg 


Heiko Sibberns, IEA Hamburg Director 

Oliver Neuschmidt, Senior Research Analyst, Unit Head, International Studies 
Milena Taneva, Senior Research Analyst, Project Co-Manager, PIRLS Data Processing 
Juliane Hencke, Senior Research Analyst, Project Co-Manager, PIRLS Data Processing 
Sebastian Mayer, Research Analyst, Deputy Project Manager, PIRLS Data Processing 
Mark Cockle, Research Analyst, Deputy Project Manager, PIRLS Data Processing 
Yasin Afana, Research Analyst 

Alena Becker, Research Analyst 

Clara Beyer, Research Analyst 

Christine Busch, Research Analyst 

Tim Daniel, Research Analyst 

Limiao Duan, Programmer 

Eugenio Gonzalez, Senior Research Analyst 

Michael Jung, Research Analyst 

Deepti Kalamadi, Programmer 

Hannah Kohler, Research Analyst 

Kamil Kowolik, Research Analyst 

Sabine Meinck, Senior Research Analyst, Head of Research, Analysis & Sampling Unit 
Ekaterina Mickheeva, Research Analyst 

Dirk Oehler, Research Analyst 

Duygu Savasci, Research Analyst, Sampling Team 

Sabine Tieck, Research Analyst, Sampling Team 

Meng Xue, Unit Head, Software 
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Statistics Canada 

Sylvie LaRoche, Senior Methodologist 

Marc Joncas, Senior Methodologist (through 2015) 
Ahmed Almaskut, Methodologist 

Shou Xiang Chen, Methodologist (through 2016) 


Educational Testing Service 


Edward Kulick, Research Director 

Jonathan Weeks, Associate Research Scientist 
Timothy C. Davey, Research Director Special Projects 
Sandip Sinharay, Principal Research Scientist 

Scott Davis, Senior Research Data Analysis Consultant 


Sampling Referee 
Keith Rust, Vice President and Associate Director of the Statistical Group Westat, Inc. 


PIRLS 2016 Reading Development Group 


Julian Fraillon Jenny Wiksten Folkeryd 

Australian Council for Educational Research Uppsala University 

Australia Sweden 

Jan Mejding Ahlam Habeeb Msaiger 

Department of Education Abu Dhabi Education Council 

Aarhus University United Arab Emirates 

nia Marian Sainsbury, Chief Reading Consultant 
Galina Zuckerman National Foundation for Educational Research 
Psychological Institute United Kingdom 

Russian Academy of Education Donald Leu 


Rese ener University of Connecticut 


Elizabeth Pang United States 
Curriculum Planning and Development Karen Wixson 
Division University of North Carolina, Greensboro 
Ministry of Education United States 
Singapore 
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PIRLS 2016 Item Development Task Force 


Prue Anderson 


Australian Council for Educational Research 
Australia 


Marian Sainsbury, Chief Reading Consultant 


National Foundation for Educational Research 
United Kingdom 


Liz Twist 


National Foundation for Educational Research 
United Kingdom 


Karen Wixson 


University of North Carolina, Greensboro 
United States 


PIRLS 2016 Questionnaire Development Group 


Joanne Latourelle 

Coordonnatrice aux études pancanadiennes 
et internationationales, Sanction des études, 
Ministére de Education, du Loisir et du 
Sport 

Canada 


Hwa Wei Ko 
Graduate Institute of Learning and Instruction 


National Central University 
Chinese Taipei 


Marc Colmant 
Ministére de léducation nationale 
Direction de lévaluation, de la prospective et 


de la performance 
France 


Maryam A. Al-Ostad 


National Centre for Education Development 
Kuwait 


Megan Chamberlain 
Comparative Education Research Unit 


Ministry of Education 
New Zealand 


Joao Maroco 


Instituto de Avaliacao Educativa, I.P. 
Portugal 


Sarah Howie 
Centre for Evaluation and Assessment 


University of Pretoria 
South Africa 


PIRLS 2016 National Research Coordinators 


Canada 

Kathryn O'Grady 

Tanya Scerbina 

Pierre Brochu (through 2016) 
Mélanie Labrecque (through 2015) 
Council of Ministers of Education 


Chinese Taipei 

Hwa Wei Ko 

Graduate Institute of Learning and Instruction 
National Central University 
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Denmark 

Jan Mejding 

Aarhus University Department of Education 
(DPU) 


Georgia 

Natia Andguladze 

Nutsa (Magda) Kobakhidze (through 2013) 
National Assessment and Examination Center 
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Ireland 
Eemer Eivers 
Educational Research Centre 


Israel 

Inbal Ron-Kaplan 

National Authority for Measurement and 
Evaluation in Education (RAMA) 


Italy 

Laura Palmerio 

Elisa Caponera (through 2013) 

INVALSI - Istituto Nazionale per la 
Valutazione del Sistema Educativo di 
Istruzione e di Formazione 


Norway 

Egil Gabrielsen 

National Centre for Reading Education and 
Research 

University of Stavanger 


Portugal 

Joao Maroco 

Ana Sousa Ferreira (through 2013) 
Instituto de Avaliacao Educativa, I.P. 


Singapore 

Elizabeth Pang 

Chan Lee Shan 

Ng Huey Bian (through 2015) 

Curriculum Planning and Development 
Division 

Ministry of Education 

Slovenia 

Marjeta Doupona 

Educational Research Institute 
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Sweden 

Agnes Tongur 

Elina Ekberg 

Tomas Matti (through 2014) 

Swedish National Agency for Education 
(SKOLVERKET) 


United Arab Emirates 

Moza Rashid AlGhufli 

Ayesha Ghanim Khalfan Almerri (through 
2016) 

Nada Abu Baker Husain Ruban (through 2015) 

Maryam Mohammed Sulaiman (through 2015) 

Aljawhara Ali AlSebaiei (through 2014) 

Assessment Department 

Ministry of Education 


United States 

Sheila D. Thompson 

National Center for Education Statistics 
U.S. Department of Education 


Benchmarking Participants 

Abu Dhabi, United Arab Emirates 

Shaikha Ali Al Zaabi 

Nasreen Hussain Al Marzoogi (through 2013) 
Ahlam Habeeb Msaiger 

Assessment Department 

Abu Dhabi Education Council 


Dubai, United Arab Emirates 

Mariam Al Ali 

Rabaa AlSumaiti (through 2014) 

Knowledge and Human Development 
Authority 
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